In addition to writing JSON according to the pipeline protocol for low-code development, MaaFramework also supports various ways to write custom code for implementing tasks:
-
FFI (Foreign Function Interface)
MaaFramework provides C and Python API, and in theory, it supports all major languages.
However, it is currently only adapted to C and Python. You may need to write other languages by yourself (we also welcome you to submit a PR after completing it!).
Advantages: Relatively high execution efficiency, more in line with standard programming paradigms. -
In simple terms, developers can write their own executable (exe) and pass the exe path through MaaToolkit interfaces. When MaaFramework reaches the corresponding task, it calls the exe, passing in the current screen's image, recognization result, task name, and other information as parameters.
Developers can perform any operation in their exe and can output commands directly (print / std::cout / ...) following the protocol. MaaFramework captures these output commands through a pipeline to execute actions such as clicking, sliding, screenshot recognition, etc. The results are then passed back through the pipeline, which can be obtained through standard input (input / std::cin / ...).
The term "exe" here refers to any executable file, including exe, bat, shell, Python scripts, etc. (also supports executable files under Linux / macOS).
Advantages: Easy to implement.
You can implement custom actions and custom recognizers, and pass them to registration interfaces.
The SyncContext can be used to invoke more clicking, sliding, screenshot recognition, etc.
-
C Language
-
Python
For custom recognizers, the launch parameters are:
/path/to/my_recognizer.exe custom_arg_1 custom_arg_2 ... sync_context image_path task_name custom_recognition_param-
/path/to/my_recognizer.exeThe path to the executable file passed through the registration interface. If it is a script like Python, you can also directly pass
Python.exe. -
custom_arg_1 custom_arg_2 ...Custom parameters passed through the registration interface (multiple parameters). If it is a script like Python, the first parameter is the path to your
.pyfile. -
sync_contextInformation needed when outputting commands to MaaFramework according to the protocol. See Input-Output Protocol for details.
-
image_pathFile path of the current screen's screenshot image.
-
task_nameThe name of the task currently being executed.
-
custom_recognition_paramThe value of
custom_recognition_paramdefined in the pipeline JSON.
For custom actions, the launch parameters are:
/path/to/my_action.exe custom_arg_1 custom_arg_2 ... sync_context task_name custom_action_param cur_box cur_rec_detail-
/path/to/my_action.exeThe path to the executable file passed through the registration interface. If it is a script like Python, you can also directly pass
Python.exe. -
custom_arg_1 custom_arg_2 ...Custom parameters passed through the registration interface (multiple parameters). If it is a script like Python, the first parameter is the path to your
.pyfile. -
task_nameThe name of the task currently being executed.
-
sync_contextInformation needed when outputting commands to MaaFramework according to the protocol. See Input-Output Protocol for details.
-
custom_action_paramThe value of
custom_action_paramdefined in the pipeline JSON. -
cur_boxThe current target position recognized by the task recognizer. Format is JSON array [x, y, w, h].
-
cur_rec_detailDetailed information recognized by the task recognizer. Format is JSON, specific protocol to be added~.
All commands executed by the controller proxy are performed synchronously. After each command is executed, the standard input (input / std::cin / scanf / ...) must be used to retrieve the execution results; otherwise, it may affect the execution of subsequent commands.
The command format is JSON with the following general structure:
-
function: string
The executed command, mandatory. Possible values:RunTask|RunRecognition|RunAction|Click|Swipe|PressKey|InputText|TouchDown|TouchMove|TouchUp|Screencap -
sync_context: string
Instance information, mandatory. When launching a custom recognizer/custom action, a string passed in as a launch parameter.
The execution result format is JSON with the following general structure:
{
"return": bool,
// ......
}return: bool
Whether the execution was successful or not.
Execute a specific sub-task.
This command requires additional parameters with the following structure:
{
"function": "RunTask",
"sync_context": string,
"task_name": string,
"task_param": object
}-
task_name: string
Task name, mandatory. -
task_param: string
Additional parameters for the task, optional.
The execution result is:
{
"return": bool
}Execute the recognition part of a specific task.
This command requires additional parameters with the following structure:
{
"function": "RunRecognition",
"sync_context": string,
"image": string,
"task_name": string,
"task_param": object
}-
image: string
Path to the image file to be recognized, mandatory. -
task_name: string
Task name, mandatory. -
task_param: string
Additional parameters for the task, optional.
The execution result is:
{
"return": bool,
"box": [int, int, int, int],
"detail": any
}Execute the action part of a specific task.
This command requires additional parameters with the following structure:
{
"function": "RunAction",
"sync_context": string,
"task_name": string,
"task_param": object,
"cur_box": [int, int, int, int],
"cur_rec_detail": any
}-
task_name: string
Task name, mandatory. -
task_param: string
Additional parameters for the task, optional. -
cur_box: array<int, 4>
Simulated range recognized, mandatory. -
cur_rec_detail: any
Simulated recognition details, mandatory.
The execution result is:
{
"return": bool
}Execute a click operation.
This command requires additional parameters with the following structure:
{
"function": "Click",
"sync_context": string,
"x": int,
"y": int
}The execution result is:
{
"return": bool
}Execute a swipe operation.
This command requires additional parameters with the following structure:
{
"function": "Swipe",
"sync_context": string,
"x1": int,
"y1": int,
"x2": int,
"y2": int,
"duration": int
}The execution result is:
{
"return": bool
}Execute a key press operation.
This command requires additional parameters with the following structure:
{
"function": "PressKey",
"sync_context": string,
"keycode": int
}The execution result is:
{
"return": bool
}Execute a text input operation.
This command requires additional parameters with the following structure:
{
"function": "InputText",
"sync_context": string,
"input_text": string
}The execution result is:
{
"return": bool
}Execute a touch-down operation.
This command requires additional parameters with the following structure:
{
"function": "TouchDown",
"sync_context": string,
"contact": int,
"x": int,
"y": int,
"pressure": int
}The execution result is:
{
"return": bool
}Execute a touch-move operation.
This command requires additional parameters with the following structure:
{
"function": "TouchMove",
"sync_context": string,
"contact": int,
"x": int,
"y": int,
"pressure": int
}The execution result is:
{
"return": bool
}Execute a touch-up operation.
This command requires additional parameters with the following structure:
{
"function": "TouchUp",
"sync_context": string,
"contact": int
}The execution result is:
{
"return": bool
}Execute a screenshot operation.
{
"function": "Screencap",
"sync_context": string
}The execution result is:
{
"return": bool,
"image": string
}Get latest screenshot without screencap.
{
"function": "CachedImage",
"sync_context": string
}The execution result is:
{
"return": bool,
"image": string
}
{ "function": string, "sync_context": string, // ...... }