File pointers

Some agents need to read files from a configured location or write files into one.

For that, agents use a common pointer structure.

origin:
  type: task
  dir: test_route.public.people
  file: people.json

destination:
  type: fs
  dir: "/tmp/somepath"
  file: people.json

The same structure is used for origin, destination and single_destination.

Structure

Element	Purpose
`type`	Pointer type. Supported values: `task`, `fs`
`dir`	Directory definition. Meaning depends on pointer type
`file`	File name

task means that the file is located and can be found by task_id

origin:
  type: task
  file: people.json

If dir is missing, current task directory is used.

origin:
  type: task
  dir: test_route.public.people
  file: people.json

For task, dir must be task-id-like value:

{route}.{schema}.{table}

Example:

test_route.public.people

fs means that the file is located in a filesystem directory.

origin:
  type: fs
  dir: "/tmp/somepath"
  file: people.json

Agents may use pointers as origin, destination, single_destination, or a combination of them.

origin usually defines where the agent reads a file from.

origin:
  type: fs
  dir: "/tmp/somepath"
  file: people.json

destination or single_destination usually defines where the agent writes a file to.

destination:
  type: task
  file: people.json

single_destination:
  type: task
  file: people.json

For some agents, destination.file may be optional. In that case, origin file name can be reused.

Example:

- do: filecopy
  origin:
    type: fs
    dir: "/tmp/somepath"
    file: people.json

  destination:
    type: task

This copies people.json into current task directory and keeps the same file name.

task is for files referenced by task_id and kept somewhere, write and read parties know only task_id
fs is for files in external filesystem folders, write and read parties must both know path
task.dir is task-id-like value.
If task.dir is missing, current task is used.
fs.dir is a real filesystem directory.
fs.dir is required.
File-specific rules may depend on the agent. For example, some agents require origin.file, while filecopy can reuse origin file name for missing destination.file.
Destination pointer may be named destination or single_destination, depending on the agent.