Flow and Action Functions
Overview
Oryonix is built around two fundamental primitives: Flows and Actions. These primitives allow you to structure your application logic in a way that allows you to take full advantage of Oryonix's durability and orchestration capabilities.
As discussed in Introduction to Oryonix, Oryonix divides Python functions into three categories.
Flow Functions
Flow functions implement complex processes or business logic that may span multiple steps, involve waiting for external events, or require coordination between different tasks. They may orchestrate the execution of tasks and manage state within a process, but they do not generally do anything themselves that interacts with the outside world. Instead, they call action functions to perform specific operations.
Flow functions are decorated with the @onix.flow decorator.
Action Functions
Action functions perform specific operations that are non-deterministic or have observable side effects. Action functions may call regular Python functions, but cannot call other action or flow functions.
Action functions are decorated with the @onix.action decorator.
Regular Python Functions
Not every function in an Oryonix service needs to be a flow or action function. Regular Python functions, and even third-party pure-Python modules and packages, may be mixed and matched with flow and action functions as needed. They have no special durability properties and are not retried on failure.
How Action and Flow Functions Work Together
Action functions are the building blocks that perform the actual work, such as making API calls, processing data, or interacting with databases. Flow functions call action functions to perform these operations, and the results can influence the flow's subsequent decisions and actions.
Action functions are for interacting directly with the world, and flow functions are for deciding when and how to interact with the world.
Make Action Functions Small and Focused
In general, it is better to err on the side of defining more, smaller action functions rather than fewer, larger ones. For example, a single database transaction might be an action function (if properly isolated), or a single REST API call might be an action function. If you find yourself needing to call an action function from another action function, that is a sign that you should probably be defining a flow function to orchestrate those action functions instead.
Separate Different Operations into Different Action Functions
You should also separate different kinds of operations into different action functions. For example, if you have a workflow that needs to read from a database, make an API call, and then write to the database, it would be better to have three separate action functions for each of those operations rather than one big action function that does all three. This way, if there is a transient failure in the middle of the workflow, only the action function that was executing at the time of the failure would need to be retried, rather than the entire workflow.
An exception to this rule is if the whole set of operations needs to be retried if any one of them fails. Continuing with the previous example, suppose the database update fails. If, due to that failure, you need to retry the whole sequence of steps--the read, the API call, and the update--then it would be better to combine all three steps into a single action function. This way, if any step fails, the entire action function will be retried, and you can avoid writing retry logic yourself.
Generate IDs and Timestamps Separately from Their Use
You should place operations that generate IDs or timestamps into their own action functions, separate from operations that use those IDs and timestamps. This helps with ensuring idempotency--if an ID or timestamp generation is retried, you will get a different ID or timestamp on each retry. Separating generation from use ensures that the subsequent operations will use consistent IDs/timestamps on each retry. If an ID were generated in the same action function as the operation that uses that ID, then if the action function were retried, a new ID would be generated and the subsequent operation would use that new ID, which could lead to unintended duplication.
Action Functions in Depth
Action Functions are the basic building blocks of an Oryonix service. Anything that interacts with the outside world must be defined as an action function. In general, if you can call the same function twice with the same arguments, and it might return different results or have outside impacts, that function must be defined as an action function.
Here are some examples of operations that should be written as action functions:
- Querying and/or updating a database
- Making an API call to another service
- Getting the current date/time
- Generating a random number or UUID
Failure Characteristics
If an action function fails due to a transient failure (e.g. a temporary network issue, a database timeout, etc.), Oryonix will automatically retry it until it succeeds. If it fails due to a non-transient failure (e.g. invalid input, an unhandled exception in the code, etc.), the failure will be propagated back to the caller.
Because of this, action functions must have the same effect no matter how many times they are called with the same input. In other words, action functions have to be idempotent in their observable effects. However, they are not required to be idempotent if the only observable effect is the return value.
For example, an action function that returns the current date and time does not need to be idempotent. Even though its return value will be different each time it is called, it does not have any other effects that would be different on each call. In Oryonix, the action function's caller would only observe the last return value, and would not be able to tell that the function had been called multiple times.
For another example, an action function that inserts a row into a database must take care not to insert duplicate rows if it is retried. Otherwise, the caller might be able to observe multiple rows being inserted into the database, which is probably not expected. This could be implemented by accepting a unique ID for each row as an argument to the action function, and using that ID as a unique index or primary key within the database.
Or, in the case of calling an external API, an action function could take an idempotency key as an argument and pass that along to the external API. This way, if the action function is retried, the external API will recognize the same idempotency key and will not perform the same action multiple times.
Execution Environment
Each action function runs in its own isolated environment. This means that action functions do not share memory with each other or with flow functions, so any modifications to global variables or in-memory state in one instance of an action function will not be visible to any other instance, or to any flow functions. Indeed, such changes will be lost even on retries of the same instance of an action function.
Moreover, an action function may not even run on the same physical machine as the flow function that called it. Oryonix may choose to run action functions on different machines for load balancing and scalability reasons.
Action functions can only communicate through their input parameters and return values, or by making external calls to databases, other APIs, etc.
Flow Functions in Depth
Flow functions are the "glue" of an Oryonix service. That is, they are higher-level functions that are typically used to implement business logic and end-to-end workflows. All API endpoints for an Oryonix service must be implemented as flow functions, and these flow functions can then call action functions or other flow functions to perform the actual work.
This means that flow functions can be used in a wide variety of situations, including:
- Implementing API endpoints
- Orchestrating multi-step workflows
- Coordinating parallel execution of action functions
- Waiting for and reacting to external events (described later)
Failure Characteristics
Unlike action functions, if a flow function experiences a transient failure (e.g. a temporary network issue, a failure of the underlying infrastructure, etc.), Oryonix will resume the flow function right from where it left off. This property is known as "durable execution".
Durable execution of flow functions is possible because Oryonix logs all operations performed by the flow function (such as calls to and values returned from action functions). If a failure occurs, Oryonix can restore the flow function's state by restarting the function and replaying all logged operations from the beginning to bring it back to the exact point it was at when the failure occurred.
Because of the need for durable execution, flow functions must be deterministic. This means that given the same inputs, a flow function must always perform the same sequence of actions and produce the same outputs. If a flow function is non-deterministic, then it may behave differently during log replay, which would lead to inconsistent behavior.
Unlike other durable execution systems, Oryonix does not require you to write your code in a special way to achieve durability. Instead, Oryonix runs flow functions in a tightly-controlled environment which makes non-determinism impossible.
Execution Environment
Just like action functions, each flow function runs in its own isolated environment. Flow functions do not share memory with other flow functions or action functions, so modifications to global variables (and other shared state) will not be visible to other flow functions or even to retries of the same flow function.
Flow functions may call other flow and action functions synchronously and/or asynchronously. They may also communicate with each other through signals. All of these features will be described in the following documents.