Custom service

With Liteflow you can create your own services that exposes their own tasks.

A service is a component of your application that can be used for different workflows and can also be shared with other projects.

Services are code that can receive tasks from a workflow.

The types of services that can be created is totally up to each developer. It’s the responsibility of the developer to decide which tasks should go to which service

Services run in Docker to provide a sandbox and a normalized environment to remove any side effects that may occur when running on many different machines. See more information on the Docker website.

Stateless service Services should be stateless, meaning that they shouldn't store any data on disk. Services should be able to restart or be replicated without losing any information or side effect. More information

Service structure

In order to make a source code compatible with Liteflow you need to add a liteflow.yml file at the root of your service.

.
├── .liteflow.yml
├── .liteflowignore
└── ... service source code

A service can be written in many languages, but it is recommended to use Javascript as it is the easiest way to get started.

When you create a service you can use the command liteflow service:init to generate a default service in Javascript or Typescript

Service definition

To define a Service, you will need to create a specific folder with a liteflow.yml file that describes its functionalities. This file can contain the following information in a YAML syntax:

You can create a default file using the CLI by entering the command:

liteflow service:init

This will create a liteflow.yml file in your current directory with the following attributes:

AttributeTypeDescription
nameStringName of the service to easily identify it.
descriptionString(optional) Description of the service.
tasksmap<id,Task>Tasks that the service can process.
configurationConfiguration(optional) Advanced configuration for the service.
repositoryString(optional) The URL of the repository (eg: https://github.com/org/repo)
name: my-service-name
description: "Description of the service"
tasks:
  my-task-a: ...
  my-task-b: ...
configuration: ...
repository: https://github.com/org/repo

Task

Tasks workflow given inputs and return specific outputs. Tasks can be called from workflows.

AttributeTypeDescription
nameString(optional) If the name of the task is not set, the name will be the ID of the task.
descriptionString(optional) Description of the task: what the task is doing and why it is useful.
inputsmap<id,Parameter>Map of inputs that the task needs in order to be executed.
outputsmap<id,Output>Map of outputs that the task returns.
name: my-task-a
description: "Description of the task"
inputs:
  input-a: ...
  input-b: ...
outputs:
  output-a: ...
  output-b: ...

Data

Describes data that can either be one of the inputs of a task, one of the outputs.

AttributeTypeDescription
nameString(optional) Name of the data
descriptionString(optional) Description of the data
type'Object' or 'String' or 'Boolean' or 'Number' or 'Any'Type of data
objectmap<string,Data>(optional) Nested data. Data can contain child data. It can only be defined when type is Object
optionalBoolean(optional) Mark the data as optional (default = false)
repeatedBoolean(optional) Define this data as an array of the type selected (default = false)
name: data-name
description: "An array of object"
type: Object # can be "String", "Boolean", "Number" or "Any"
optional: false
repeated: true # make the data iterable (eg: an array)
object:
  nested-data-a: ...
  nested-data-b: ...

Configuration

Configuration is used for advanced configuration over your service. This is totally optional and you should be careful while using this as it might impact the security of your service.

AttributeTypeDescription
envString[](optional) List of environmental variables needed for the dependency with the format KEY=VALUE.
commandString(optional) The command to run when the Service starts.
argsString[](optional) Arguments to pass to your command.
env:
  - FOO=BAR
command: "my-program"
args: ["args", "for", "my", "program"]

Full example

liteflow.yml
name: my-service-name
description: "Description of the service"
tasks:
  my-task-a:
    name: my-task-a
    description: "Description of the task a"
    inputs:
      input-a:
        name: data-name
        description: "An array of object"
        type: Object
        repeated: true
        object:
          nested-data-a:
            type: String
          nested-data-b:
            type: Number
      input-b:
        type: String
    outputs:
      output-a:
        type: String
      output-b:
        type: Number
  my-task-b:
    name: my-task-b
    description: "Description of the task b"
    inputs:
      input-a:
        type: Number
    outputs:
      output-a:
        type: Boolean
configuration:
  env:
    - FOO=BAR
  command: "my-program"
  args: ["args", "for", "my", "program"]
repository: https://github.com/org/repo

Create a task

Services need to receive a request for an execution from a workflow to execute any desired tasks. Every time a request for execution is received, the service is automatically called with the correct function.

The service doesn't need to handle any communication to be able to process a task.

Handle executions

First, make sure to describe your task in the liteflow.yml file in the tasks section.

The service needs to start listening for tasks by calling the function listenTask from the @liteflow/service library.

This function accepts an object that associates a function to a task name (as defined in your liteflow.yml).

The function accepts in argument the inputs as describe in the liteflow.yml and should return an object containing the data described in liteflow.yml.

Example

index.js
const Service = require("@liteflow/service");
const liteflow = new Service();

liteflow.listenTask({
  taskX: (inputs) => {
    // inputs match the definition in the liteflow.yml
    // Can directly throw error
    if (inputs.inputX === undefined) throw new Error("inputX is undefined");
    // Return an object that should match the definition in the liteflow.yml
    return {
      foo: "test",
      bar: true,
    };
  },
  // tasks can by asynchronous using async/await
  asyncTask: async (inputs) => {
    const result = await myFunctionWithPromise();
    return result;
  },
});

See the @liteflow/service library for additional documentation

Execute a task

This type defines the task to execute of a given instance of a service.

By default, the task's inputs are the previous step's outputs. It can be customized by defining the inputs parameter to reference the outputs of any previous steps.

Definition

KeyTypeDescription
type"task"
keyString(optional) Key to identify this step
instanceInstanceInformation about the instance of the service to run.
taskKeyStringKey of the task to execute
inputsmap<string,Inputs>Key of the task to execute

Instance

KeyTypeDescription
srcStringSource of the service to deploy
envString[](optional) Environment variable to use for the service