​
set_logger

logger
logging.Logger
required

Sets a logger for this Node. Throws error if not a valid logger.

​
get_status

Returns the status of the node’s execution. Will be one of PENDING, EXECUTING, SUCCESS, FAILED.

Example: SUCCESS

​
get_id

ID of this node.

​
get_name

Name assigned to the instance of this node.

​
get_input

Returns input as a dictionary of field name keys and type values. Input is meant to represent values passed in and set dynamically from other nodes.

​
get_output

Returns output as a dictionary of field name keys and type values.

​
get_execute_args

Returns execute args as a dictionary of field name keys and type values. Execute args is meant to be set at runtime before a DAG starts.

​
safe_get_execute_args

key
str
required

The key that being retrieved from execute args.

default
str

The expected default value in case the key is not found. Defaults to None.

Returns execute args as a dictionary of field name keys and type values. Execute args is meant to be set at runtime before a DAG starts.

​
set_status

status
str
required

Set the status of the node’s execution. Must be one of PENDING, EXECUTING, SUCCESS, FAILED. If not, will raise an error.

​
set_input

input
dict[str:type]
required

Set input values to this Node as a dictionary. Input is meant to represent values passed in and set dynamically from other nodes.

​
set_output

output
dict[str:type]
required

Set output values to this Node as a dictionary. Output is the output of this node.

​
set_execute_args

*args
list[type]
required

Set *args to execute_args. Execute args are meant to be set at runtime before a DAG starts.

**kwargs
dict[str:type]

Set **kwargs to execute_args.

​
set_input_s

key
str
required

Set input schema as a dictionary of field name keys and type values. Input is meant to represent values passed in and set dynamically from other nodes.

The allowed schema definitions are as follows:

  • The schema must be a dictionary 
    Schema = 
    {
        "outer": str,
        "inner": {
            "level": int
        }
    }
  • Lists must be denoted as literals, and are treated as a set of allowable values
  • Dictionaries must have strings as keys, and types as values
  • Other types must be from the following selection: int, float, str, bool, bytes, dict (generic), list (generic).
  • Parametrized generics are not supported as of yet and will throw an exception
  • Custom classes are not supported as of yet and will throw an exception
    Schema = {
        "name": str,
        "favorites": dict, # will allow any dict containing anything
        "cities_lived": list, # will allow any list containing anything
        "schools_attended": [str], # will allow any strings
        "pre_ipo_user": bool,
        "age": float,
        "secret_code": bytes
    }

​
set_output_s

key
str
required

Set output schema as a dictionary of field name keys and type values. Output is the output of this node.

The allowed schema definitions are as follows:

  • The schema must be a dictionary 
    Schema = 
    {
        "outer": str,
        "inner": {
            "level": int
        }
    }
  • Lists must be denoted as literals, and are treated as a set of allowable values
  • Dictionaries must have strings as keys, and types as values
  • Other types must be from the following selection: int, float, str, bool, bytes, dict (generic), list (generic).
  • Parametrized generics are not supported as of yet and will throw an exception
  • Custom classes are not supported as of yet and will throw an exception
    Schema = {
        "name": str,
        "favorites": dict, # will allow any dict containing anything
        "cities_lived": list, # will allow any list containing anything
        "schools_attended": [str], # will allow any strings
        "pre_ipo_user": bool,
        "age": float,
        "secret_code": bytes
    }

​
set_pre_execute_hook

hook
Callable[[dict[str:type]], dict[str:type]]
required

Set a hook to be called before the node executes within the DAG.

Can be used to hard-set certain inputs or any custom functionality such as logging inputs to analytics.

​
set_post_execute_hook

hook
str
required

Set a hook to be called after the node executes within the DAG. TODO: what is this meant to be used for?

Log outputs to analytics, send to data warehouse, CRM, etc

​
validate_input

validate_input gets called in dag.execute, but feel free to call it wherever you want.

Whether or not the input data (input) matches the input schema (input_s). Although we raise an exception for the failure case in DAG, whether you want to raise an exception or not when you use the method otherwise is up to you.

​
validate_output

validate_output gets called in dag.execute, but feel free to call it wherever you want.

Whether or not the output data (output) matches the output schema (output_s). Although we raise an exception for the failure case in DAG, whether you want to raise an exception or not when you use the method otherwise is up to you.

​
validate_execute_args

Whether or not the execute args data (execute_args) matches the execute_args schema (__execute_args_s).

​
execute

Abstract method which must be implemented in a subclass. This method is called when the node is executed within a DAG.

In order to access the input for the Node, use self.get_input(). Once you have created the output dictionary, instead of returning it, set it as the Node’s output by using self.set_output().

Run your DAGDAG