# Welcome to AugeLab Studio User Manual

1\.	Introduction

This manual is created for both newcomers and seasoned users.

Following this documentation, you'll learn everything related to AugeLab Studio and automation, without needing to write code or having a related background or experience.

We recommend it reading until the **Getting Started** section and then refer to the **Further Reading**.


# AugeLab Studio

<figure><img src="/files/cxlhxkogsYAY3iOTel1v" alt=""><figcaption></figcaption></figure>

AugeLab Studio is a cutting-edge cross-platform, agentic no-code platform designed for seamless vision applications and integration.

<figure><img src="/files/WH14iNG2WfRCO9KwPxi9" alt=""><figcaption><p>Example Usage</p></figcaption></figure>

A simple workflow just like the one below has input reading, image processing and industrial communication!

<figure><img src="/files/WFoZ4YmJMwKtTw7apQb9" alt=""><figcaption><p>Simple Scenario</p></figcaption></figure>

Whether you're a beginner or an expert, AugeLab Studio offers the tools and flexibility to bring your vision projects to life.


# Key Features

AugeLab Studio is a no-code AI powered industrial vision platform. It lets you acquire images, process them, make decisions, train AI models, communicate with hardware, and deploy the result from one workspace.

<figure><img src="/files/WFoZ4YmJMwKtTw7apQb9" alt="AugeLab Studio Workspace"><figcaption><p>AugeLab Studio visual workflow editor</p></figcaption></figure>

## Why No-Code?

Build scenarios by connecting function blocks instead of writing application code. A single workflow can include camera input, image processing, logic, math, hardware communication, and output actions.

Use local inference and training with your own datasets; object detection, tracking, pose estimation, segmentation, OCR, barcode reading, measurement, and anomaly-style inspection.

Connect industrial cameras and factory systems such as Basler, Hikvision, iTek, iRayple, GigE Vision, GenICam, RTSP, USB cameras, PLCs such as Siemens S7, OPC UA, Modbus, MQTT, REST API, Email, and SMS.

## Main Tools

<table data-view="cards"><thead><tr><th>Feature</th><th>What it helps you do</th><th data-card-target data-type="content-ref">Open</th></tr></thead><tbody><tr><td>AI Assistant</td><td>Ask for workflow ideas, block explanations, troubleshooting help, and custom logic.</td><td><a href="/pages/1FfSeZtsliZSVQ0ktf4u">/pages/1FfSeZtsliZSVQ0ktf4u</a></td></tr><tr><td>Annotation</td><td>Collect images, label objects, use assisted annotation, and prepare datasets.</td><td><a href="/pages/Lagyme7wyJdJObK7S6us">/pages/Lagyme7wyJdJObK7S6us</a></td></tr><tr><td>Model Training</td><td>Train object detection models with your own dataset inside AugeLab Studio.</td><td><a href="/pages/yo1aRPIAZ1Dbyk2sayls">/pages/yo1aRPIAZ1Dbyk2sayls</a></td></tr><tr><td>Plugin Designer</td><td>Create reusable custom blocks when the built-in block library is not enough.</td><td><a href="/pages/Z3q1NWC328X3luJRIayf">/pages/Z3q1NWC328X3luJRIayf</a></td></tr><tr><td>Headless Studio</td><td>Run saved scenarios from Python, CLI, Docker, Linux services, or edge devices.</td><td><a href="/pages/wbOE6H8ADaP0myWjScbs">/pages/wbOE6H8ADaP0myWjScbs</a></td></tr><tr><td>Widget to Socket</td><td>Turn node settings into input sockets so other blocks can control parameters.</td><td><a href="/pages/WHw7LCjzUKB7Hw736fhf">/pages/WHw7LCjzUKB7Hw736fhf</a></td></tr><tr><td>Python Packages</td><td>Install extra Python packages and use them in your custom workflows.</td><td><a href="/pages/3VMfMbvroWQX2fRmRFsp">/pages/3VMfMbvroWQX2fRmRFsp</a></td></tr><tr><td>Community Sharing</td><td>Package and share solutions so others can reuse your work.</td><td><a href="/pages/lsJFbjdL2eDEMwISBDLd">/pages/lsJFbjdL2eDEMwISBDLd</a></td></tr></tbody></table>


# Use Cases

AugeLab Studio fits workflows where images need to become decisions, measurements, records, or hardware actions. These examples show common starting points.

<details>

<summary>Manufacturing and Quality Control</summary>

Inspect parts for scratches, cracks, missing components, wrong orientation, or assembly errors. Measure dimensions with sub-pixel tools and send pass/fail results to operators, PLCs, databases, or reports.

</details>

<details>

<summary>Automotive</summary>

Check paint finish, part alignment, labels, welds, connectors, and final assembly state. Vision workflows can also support test rigs and safety research such as lane, obstacle, or fatigue detection.

</details>

<details>

<summary>Safety and Workplace Monitoring</summary>

Detect missing PPE, restricted-zone entry, unsafe machine access, falls, or long immobility. Connect results to lights, alarms, PLC signals, dashboards, or messages.

</details>

<details>

<summary>Robotics and Logistics</summary>

Locate objects for pick-and-place, sorting, welding, packing, and routing. Read barcodes, classify packages, and trigger lane or conveyor actions from the same scenario.

</details>

<details>

<summary>Healthcare and Research</summary>

Support controlled research workflows such as counting cells, analyzing microscope images, measuring structures, or highlighting areas of interest. Clinical use requires validation under the user's own regulatory process.

</details>

<details>

<summary>Agriculture</summary>

Use camera and drone images to inspect crop health, detect disease patterns, estimate counts, or monitor livestock movement and behavior.

</details>

<details>

<summary>Security and Retail</summary>

Monitor access, crowd density, shelf state, missing products, queue length, or unusual activity. Results can trigger alerts, logs, reports, or restock actions.

</details>

These are examples, not limits. If the task can be described as visual input, decision logic, and an output action, it can usually be prototyped in AugeLab Studio.


# System Requirements

For Image Processing Applications

AugeLab Studio is widely used on desktops to edge devices with varying operating systems. Below, you may refer to the **minimum requirements** of hardware and operating systems:

| Hardware         |                    Basic                    |                      AI                     |
| ---------------- | :-----------------------------------------: | :-----------------------------------------: |
| Processor        |         i3 10th generation processor        |         i5 10th generation processor        |
| Memory           |                     8GB                     |                     16GB                    |
| Graphics Card    |            Internal Graphics Card           |                NVidia GTX1650               |
| Operating System |        Windows 10 / Linux Glibc 2.31        |        Windows 10 / Linux Glibc 2.31        |
| Storage          |             3GB available space             |             3GB available space             |
| Network          | Internet connection required for activation | Internet connection required for activation |

{% hint style="info" %}
Tested linux systems are:

* Ubuntu 20.04 or higher
* Debian 11 or higher
* Raspberry pi 5 with Raspberry Pi OS (64-bit)
* Jetson Orin with jetpack 6.2 or higher
  {% endhint %}

{% hint style="warning" %}
Please remember to look at the [Cuda Compute Capability](https://developer.nvidia.com/cuda-gpus) score when choosing a graphics card. **We recommend a one with at least 3.5**.
{% endhint %}

<details>

<summary>Additional Supported Platforms</summary>

Additional platforms do not have a user interface. Instead, a python API is exposed for usage.

#### ☁️ Cloud / Linux <a href="#cloud-linux" id="cloud-linux"></a>

* **Processor**: 2 vCPUs (minimum)
* **Memory**: 4GB (minimum)
* **Operating System**: tested with Ubuntu 20.04
* **Storage**: 3GB available space

#### 🖥️ Jetson Nano <a href="#jetson-nano" id="jetson-nano"></a>

* **Processor**: Quad-core ARM Cortex-A57
* **Memory**: 4GB
* **Graphics**: 128-core Maxwell GPU
* **Operating System**: Ubuntu 20.04 (64-bit)
* **Storage**: 16GB microSD card

#### 🖥️ Raspberry Pi <a href="#raspberry-pi" id="raspberry-pi"></a>

* **Processor**: Quad-core ARM Cortex-A72 (Raspberry Pi 4 or 5)
* **Memory**: 4GB or 8GB
* **Graphics**: VideoCore VI GPU
* **Operating System**: Raspberry Pi OS (64-bit)
* **Storage**: 16GB microSD card

</details>


# Signing up

You need an AugeLab account before you can download AugeLab Studio and get your verification code.

{% stepper %}
{% step %}

### Open the sign-up page

Go to [account.augelab.com](https://account.augelab.com/?p=signup) and open the Sign Up section.

<figure><img src="/files/WoazF3oJ0U5eIrV9WnTd" alt="AugeLab account sign-up form" width="250"><figcaption><p>Sign-up interface</p></figcaption></figure>
{% endstep %}

{% step %}

### Create your account

Fill out all fields, accept the Terms and Conditions, then click **Sign Up**.
{% endstep %}

{% step %}

### Confirm your email

A confirmation code will be sent to your email address. Copy that code into the confirmation form.

<figure><img src="/files/FiHGVXZoe5S0WoBcWagz" alt="AugeLab account confirmation form" width="250"><figcaption><p>Confirmation interface</p></figcaption></figure>
{% endstep %}

{% step %}

### Open account management

After confirming your email, log in with your username and password. You will see the Account Management page.

<figure><img src="/files/GwYPdmUr3sLCgud3StfL" alt="AugeLab account management page" width="750"><figcaption><p>Account management interface</p></figcaption></figure>
{% endstep %}

{% step %}

### Save your verification code

In the **Licenses** section, find your verification code. It uses this format:

```
xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx
```

You need this code when you start AugeLab Studio for the first time. Save it somewhere safe or copy it to your clipboard.
{% endstep %}

{% step %}

### Download AugeLab Studio

Click <img src="/files/qywSZKznpP4VhV4cVzyN" alt="download studio" data-size="line"> to download the AugeLab Studio installer.
{% endstep %}
{% endstepper %}


# Installation

Choose the installation path for your operating system. After installation, continue with [Activation](#activation).

{% tabs %}
{% tab title="Windows" %}
Download AugeLab Studio from [account.augelab.com](https://account.augelab.com), then run `AugeLab_Studio.exe`.

{% hint style="info" %}
Before you start:

* Run the installer with elevated privileges.
* Keep a stable internet connection during installation and activation.
  {% endhint %}

{% stepper %}
{% step %}

## Start the installer

Run `AugeLab_Studio.exe`.

<figure><img src="/files/UYprgSZRRoPN6CqjAQXC" alt="AugeLab Studio setup wizard" width="400"><figcaption><p>Setup wizard</p></figcaption></figure>

Click **Next**.
{% endstep %}

{% step %}

## Accept the user agreement

<figure><img src="/files/UsaKiFMqOKQIgMLIXTCY" alt="AugeLab Studio user agreement" width="400"><figcaption><p>User agreement</p></figcaption></figure>

Read the agreement, then click **Agree**.
{% endstep %}

{% step %}

## Choose install options

<figure><img src="/files/Y2dd3PwsSRPzZHvqgldD" alt="AugeLab Studio install options" width="400"><figcaption><p>Install options</p></figcaption></figure>

By default, AugeLab Studio is installed for the current user. You can change the install mode or install multiple instances from this screen.

### GPU acceleration / CUDA <a href="#gpu-acceleration-cuda" id="gpu-acceleration-cuda"></a>

If you have a compatible NVIDIA GPU and want GPU acceleration, enable the GPU/CUDA option when it is shown. AugeLab Studio ships the CUDA runtime components it needs. For hardware details, see [system requirements](/introduction/system-requirements).
{% endstep %}

{% step %}

## Install and launch

Click **Install**. The installation may take up to 5 minutes depending on your hardware and internet connection.

When installation is complete, start AugeLab Studio from the desktop shortcut or Start menu.
{% endstep %}
{% endstepper %}
{% endtab %}

{% tab title="Linux" %}
Download the Linux installer script from [account.augelab.com](https://account.augelab.com), then run it from a terminal.

{% stepper %}
{% step %}

## Make the installer executable

```bash
chmod +x installer.sh
```

{% endstep %}

{% step %}

## Install Studio

Use the same installer flow for Linux x86-64 and ARM64:

```bash
./installer.sh --ui
```

For AI/GPU support, make sure the NVIDIA driver is installed and visible on the host, then run:

```bash
./installer.sh --ui --ai
```

AugeLab Studio ships the CUDA runtimes it needs. On Jetson devices, use the standard JetPack/NVIDIA driver stack for your device.
{% endstep %}

{% step %}

## Launch or uninstall

The installer creates the `augelab_studio` launcher and desktop entries when UI mode is selected.

To uninstall:

```bash
./installer.sh --uninstall
```

{% endstep %}
{% endstepper %}
{% endtab %}
{% endtabs %}

## Activation <a href="#activation" id="activation"></a>

{% stepper %}
{% step %}

## Open AugeLab Studio

Run the application. The activation window will appear.

<figure><img src="/files/UxZ1SPo1e9n7VQCaxTxi" alt="AugeLab Studio activation window" width="400"><figcaption><p>Activation window</p></figcaption></figure>
{% endstep %}

{% step %}

## Enter your verification code

Copy the code you saved during [signing up](/getting-started/signing-up), then paste it into **Enter verification code...**.

<figure><img src="/files/5zRBiRYgMNx7APtxM6c0" alt="AugeLab Studio verification code entry" width="400"><figcaption><p>Verification code entry</p></figcaption></figure>
{% endstep %}

{% step %}

## Verify

Click **Verify**. After the confirmation message appears, AugeLab Studio is ready to use.
{% endstep %}
{% endstepper %}

{% hint style="info" %}
Below, you may find the license and user agreement.

<details>

<summary>LICENSE AND USER AGREEMENT</summary>

## \*\*License\*\* <a href="#license" id="license"></a>

Important: Carefully read the rights and obligations set forth in this license agreement. During installation, you will be asked whether you accept these terms. If you do not agree, the software will not be installed on your computer. Installing the program on your computer indicates that you accept the terms of the agreement. This agreement for the AugeLab Studio Software (SOFTWARE) is a legal agreement between AugeLab and the END USER (natural or legal). By installing the SOFTWARE, you accept the terms of this agreement. If you do not accept the agreement, you cannot install and use the SOFTWARE.

{% hint style="info" %}
The software you downloaded is the Community version. It has all the features, but the runtime is limited to 15 minutes. After 15 minutes, the software stops. You can get the software to run state again with the Run button. **This license type is not suitable for commercial use**. For commercial use license, please contact <info@augelab.com>.
{% endhint %}

## \*\*RIGHTS AND OBLIGATIONS OF THE END USER:\*\* <a href="#rights-and-obligations-of-the-end-user" id="rights-and-obligations-of-the-end-user"></a>

1\. The END USER is responsible for meeting the minimum system requirements for the SOFTWARE (defined under the heading 1.3 System Requirements). It accepts, undertakes and declares to use the infrastructure services required for the operation of the SOFTWARE (Example: Internet, camera system or all hardware products belong to the company that will use the software). The END USER declares and accepts that he/she knows that the right to benefit from the provisions of clause III-1.b, 1.c of this contract will cease to exist due to the damages and losses that may occur in the product subject to the contract due to the violation of the above written provision.

2\. The END USER accepts, undertakes and declares that he will install the SOFTWARE on a single computer with a single license and use it only in these environments, and will not transfer it to another environment permanently or temporarily by any means or in any way.

3\. END USER; The SOFTWARE declares and accepts that if it supports a multi-user system, it can be installed and run on more than one computer in the network, that it must obtain a license for each computer in the network, and that a SOFTWARE license cannot be shared and the SOFTWARE cannot be used on different computers at the same time.

4\. If the SOFTWARE is used for multiple users; The END USER has the right to install the maximum program that can be run on other computers as much as the number of clients specified when obtaining the license..

5\. The END USER accepts, undertakes and declares that s/he will use the community version for the registration number and duration determined by AugeLab, and not for any reason, for commercial or professional purposes or other profit-making reasons.

6\. END USER

a. Will not reproduce the SOFTWARE and documentation other than those permitted by this License Agreement, will not decompile or recompile the SOFTWARE,

b. shall not distribute, lend, rent, sell, or transmit the Premium version of the SOFTWARE to any other person, without the written consent of AugeLab,

c. It accepts, declares and undertakes that it will not change or adapt the SOFTWARE and its documentation or create inspired works based on the SOFTWARE and documentation.

7\. AugeLab Image Processing and Industrial Automation Technologies Industry Inc. reserves all rights on this software. The right holder has the right to make paid features or change the licensing method and price in the next SOFTWARE versions or on this version.

</details>
{% endhint %}


# First Look

After the installation is finished, the shortcut to run AugeLab Studio will be created at your desktop or your start menu. Go ahead and run the application.

At first, you'll see the main dashboard and a several windows:

<figure><img src="/files/8HDfg7ZIXNziOd4N1wz3" alt="Singing-up" width="750"><figcaption><p>Usage Statistics</p></figcaption></figure>

First prompt is to ask for usage statistics. These statistics help us develop better features for you, accepting it helps us a lot!

Next, you'll see the survey page. You can take this survey any time you want from the 'Help' section.

<figure><img src="/files/TES7gAOHJ4odnhhl7h75" alt="Singing-up" width="450"><figcaption></figcaption></figure>

Following that, you'll see two new windows that shows updated features and example projects.

You may close them since they all are accessible through the menubar.


# Simple Tour

<figure><img src="/files/BWehM7YfNmNUwbQ7etyP" alt="Singing-up" width="750"><figcaption><p>AugeLab Studio User Interface</p></figcaption></figure>

AugeLab Studio is designed with simplicity and ease of learning in mind. There are only four main sections you'll be using during your AugeLab Studio journey:

<figure><img src="/files/LFh1AqJKPPHBFrhpGiZs" alt="Singing-up" width="750"><figcaption><p>AugeLab Studio User Interface</p></figcaption></figure>

1️⃣ `Menubar` is where most tools, utilities and controls lie. You create scenarios, save/load them, run and debug them using this section.

2️⃣ `Scenario Area` is where you create your own solutions. You can create a new scenario, open example projects and get back to documentation from here.

3️⃣ `Blocks Area` is where all out-of-box function blocks can be seen.

4️⃣ `Log Window` is used to log and show information, warnings and errors to the user.

5️⃣ `AI Agent` area is where you can interact with the AI Agent to get help, automate, getsuggestions and code snippets.

***

That's all! Now, click on `File ➡️ New` from the top left side of menubar and let's make your very first computer vision project!


# Your Very First Project

Since you are all set-up, let's create your very first project!

For your first project, you'll be creating a Golf Score Detector algorithm without coding a single line. We'll:

1. Detect the golf ball
2. Define where the score happens
3. Send out the result

Let's begin moving moving onto the next page below.


# Basics

In this part, you will load a video, add your first blocks, connect sockets, run one frame at a time, and select the golf ball as a reference.

{% stepper %}
{% step %}

### Download the sample video

Download the sample video below.

{% file src="/files/La3aUpMZcBZ3rffKrjGz" %}
{% endstep %}

{% step %}

### Add the Video block

Return to AugeLab Studio and drag the **Video** block from the blocks bar into the scenario.

![](/files/gs4Db1bBsRc9N54oqoeS)

Click **Select Video File** inside the block and load the video.

![](/files/b45jI0CWDvaXtGMVlcpo)

Disable real-time streaming by clicking **Real-time**.

![](/files/gZbFmPUK6xB9za2gcSMT)
{% endstep %}

{% step %}

### Add Image ROI Select

Go to **Detections/Shapes** in the blocks bar and drag **Image ROI Select** into the scenario.

![](/files/8bD4C8EYa9ZuBfiMc8Fh)

When it is first added, the block is empty because no image is connected yet.

![](/files/r555V1qUNcJryI2ORUYB)
{% endstep %}

{% step %}

### Connect the blocks

Click the **Image** output socket of the **Video** block.

![](/files/SyahtfyLuFieeAlo0pyE)

Move the cursor to the **Image** input socket of **Image ROI Select** and click again to create the connection.

![](/files/oHkpTjgv9WmtenJ3w95b)
{% endstep %}

{% step %}

### Run one frame

Press **Run Step** from the menu bar to process one video frame.

![](/files/MEEaXJqNJmG7Okmu91Tt)

You should now see the frame in the ROI block.

![](/files/ximTuhlq7nbcIIT2nti1)
{% endstep %}

{% step %}

### Select the golf ball

Click, drag, and release around the golf ball to define the ROI.

![](/files/nEtRj6QTcjsMbF7XEUcj)

This selected area will be used as the reference in the next part.
{% endstep %}

{% step %}

### Move and zoom the scenario

Press and hold the middle mouse button to move around the scenario.

![](/files/99VD5Ln1bn3pttKtKPZF)

Use the mouse wheel to zoom in and out.

![](/files/DxI5irTEEPvm5xmz3y5n)
{% endstep %}
{% endstepper %}

You now know the basics: adding blocks, connecting sockets, running a step, selecting an ROI, and navigating the scenario. Continue with the detection page.


# Detection

In this part, you will detect the golf ball position. You will first build a simple detector, then fix the reference so detection remains stable while the ball moves.

{% stepper %}
{% step %}

### Add Find Object

Go to the blocks bar. Under **Detections/Shapes**, open **Detectors** and drag **Find Object** into the scenario.

![](/files/Kz4QLPQfKYIagoHV4o8l)
{% endstep %}

{% step %}

### Add Show Image

Under **Input/Output**, open **Outputs/Exports** and drag **Show Image** into the scenario.

![](/files/zX2iS0cVeD4zdf9dXU9f)
{% endstep %}

{% step %}

### Connect the detector

Connect the blocks as shown below.

![](/files/n5HxQ6M0usu4qzTkSGPO)

Click <img src="/files/YAlVdoT2RjcfaRrTpP2p" alt="run step" data-size="line">*Run Step*. You should see the detected golf ball inside a red box in **Show Image**.

![](/files/O3oxWYKSjOQuxxgaxJ5d)
{% endstep %}

{% step %}

### Test continuous detection

Click <img src="/files/dEBSnHPh0rec7YKCQ54r" alt="run" data-size="line"> *Run* and watch the detection for about 10 seconds.

You may notice that detection becomes unstable. This happens because **Image ROI Select** keeps changing its reference as the ball moves, so the reference can become grass instead of the golf ball.
{% endstep %}

{% step %}

### Add Image Memory

To keep the reference stable, add **Image Memory**.

Go to the blocks bar. Under **Image Transformers**, open **Analysis** and drag **Image Memory** into the scenario.

![](/files/ZzwOo9oqAEL4EzPn4KPC)
{% endstep %}

{% step %}

### Add Logic Input

Under **Input/Output**, open **Data Inputs** and drag **Logic Input** into the scenario.

![](/files/9TZNPwBGATbeyEyblfhp)
{% endstep %}

{% step %}

### Rearrange crowded blocks

Move one block by clicking and dragging it.

![](/files/nyfAfYurtXQQzUbMUzAT)

Move multiple blocks by dragging a selection around them, then dragging the selected group.

![](/files/PXcgQO2dF19IAwcn6fnG)
{% endstep %}

{% step %}

### Connect Image Memory

Connect the new blocks as shown below.

![](/files/BgXCybBF5VZe4QJwTkCd)

**Image Memory** freezes the frame we need, so the reference does not change while the video runs.
{% endstep %}

{% step %}

### Save the reference frame

Run the scenario for one step. Then set **Logic Input** to **True** to save the image in **Image Memory**.

![](/files/B6z6NXne558KiqHtAYaL)
{% endstep %}

{% step %}

### Tune Find Object

Move to **Find Object** and set **Match Threshold** to `100%`.

<figure><img src="/files/C9Sq35pcXx6HnCEdLGFC" alt="Find Object match threshold" width="450"><figcaption><p>Find Object match threshold</p></figcaption></figure>
{% endstep %}

{% step %}

### Run detection

Press *Run* and check that the detector follows the golf ball.

![](/files/usmdy4wtqn8nffSNmxyt)
{% endstep %}
{% endstepper %}

The detector is now working with a stable reference. Continue to the final page to check whether the golf ball reaches the hole.


# Wrapping Up

In this part, you will check whether the detected golf ball is inside the hole area, then show the final result.

{% stepper %}
{% step %}

### Add Rectangles in Rectangle

Go to the blocks bar. Under **Detections/Shapes**, open **Roi Processing** and drag **Rectangles in Rectangle** into the scenario.

![](/files/TTkjnRxWL7Jm9MgUsVM4)

<details>

<summary>Add the block with search instead</summary>

You can add blocks faster with search. Double-click an empty area and type **Rectangles**.

<figure><img src="/files/494SPQHtsKp5Or2VlEVw" alt="Search for Rectangles in Rectangle" width="450"><figcaption><p>Search for a block</p></figcaption></figure>

Press *Enter*. The block is created under your cursor.

<figure><img src="/files/B8KsmOoKNuTztX1tfI4s" alt="Rectangles in Rectangle block" width="450"><figcaption><p>Rectangles in Rectangle block</p></figcaption></figure>

</details>
{% endstep %}

{% step %}

### Add the hole ROI

Add one more **Image ROI Select** block. Use it to select the hole area.

Rearrange and connect the blocks as shown below.

![](/files/OD6sLHRJWVjDYsZVuHgs)

**Rectangles in Rectangle** checks whether detected rectangles are inside the selected reference area.
{% endstep %}

{% step %}

### Add visual outputs

Connect **Led Output** and **Show Image** so you can see whether a score is detected.

![Led Output and Show Image connected](/files/VlRmhWHiOE5Gxm61ZN0v)
{% endstep %}

{% step %}

### Run the finished scenario

Click <img src="/files/dEBSnHPh0rec7YKCQ54r" alt="run" data-size="line"> to run the scenario.

![Finished golf scoring detector](/files/kmQ1sPOWGskL2XX7nqKn)
{% endstep %}
{% endstepper %}

You have finished your first AugeLab Studio scenario: a golf scoring detector.

You learned how to add blocks, connect sockets, select ROIs, run a scenario, stabilize a reference image, detect an object, and display a final result.


# More Local Examples

You can also find local examples with explanations under:

<figure><img src="/files/JBu6J1LdLqLGBY1RK4Uc" alt="" width="450"><figcaption></figcaption></figure>

Clicking will open a window containing few example scenarios. This window will be useful when you are stuck at a certain problem and looking for an example fix:

<figure><img src="/files/aZaj90TxDCWBzIBS4nAg" alt="" width="450"><figcaption></figcaption></figure>

Clicking any button under each section, such as `Count Objects` will open the example scenario for you to try.


# Further Reading

You finished your first AugeLab Studio project. Use these pages as your next path depending on what you want to build next.

<table data-view="cards"><thead><tr><th>Topic</th><th>Use it when you want to...</th><th data-card-target data-type="content-ref">Open</th></tr></thead><tbody><tr><td>User Interface</td><td>Move faster in the workspace and learn the main panels, menus, and scenario controls.</td><td><a href="/pages/a5zklxPzWrapk8TR9lLw">/pages/a5zklxPzWrapk8TR9lLw</a></td></tr><tr><td>More Examples</td><td>Build practical inspection workflows from finished example projects.</td><td><a href="/pages/NIwH2kMycZQ6nSex0kxr">/pages/NIwH2kMycZQ6nSex0kxr</a></td></tr><tr><td>Camera Setup</td><td>Connect cameras and choose the right camera input path for your hardware.</td><td><a href="/pages/2PYEZ42a0AXKkFzv0ARE">/pages/2PYEZ42a0AXKkFzv0ARE</a></td></tr><tr><td>Function Blocks</td><td>Find what each block does, which sockets it uses, and how to combine it with other blocks.</td><td><a href="/pages/g5eNDn4ZtnlJMV1QGPIl">/pages/g5eNDn4ZtnlJMV1QGPIl</a></td></tr><tr><td>Key Features</td><td>Explore AI tools, annotation, model training, plugins, headless mode, and other major workflows.</td><td><a href="/pages/AKyOhKSQCZnb21oX2yF2">/pages/AKyOhKSQCZnb21oX2yF2</a></td></tr><tr><td>Communication Protocols</td><td>Connect AugeLab Studio to PLCs, industrial protocols, services, and external systems.</td><td><a href="/pages/8QG8AScuyw3yG1d7v63l">/pages/8QG8AScuyw3yG1d7v63l</a></td></tr></tbody></table>

{% hint style="info" %}
If you are not sure where to go next, start with User Interface, then browse Function Blocks while building your own scenario.
{% endhint %}


# Detailed Look

<figure><img src="/files/LFh1AqJKPPHBFrhpGiZs" alt="Singing-up" width="750"><figcaption><p>AugeLab Studio User Interface</p></figcaption></figure>

Let's dive in a little bit deeper on the four main sections of AugeLab Studio:

## 1. Menubar <a href="#id-1-menubar" id="id-1-menubar"></a>

![](/files/yLz40AYlEtTNznfTn4XB)

The Menubar is located at the top of the interface and provides access to essential project operations.

You can save, load, run and analyze scenarios. Also, AI utilities are provided here.

The detailed explanation will be provided in the upcoming pages.

## 2. Scenario Area <a href="#id-2-scenario-area" id="id-2-scenario-area"></a>

![](/files/gs4Db1bBsRc9N54oqoeS)

The Scenario Area is the central workspace where you design and build your image processing workflows. In this area, you can drag and drop different blocks to create a sequence of operations.

Each block represents a specific function or algorithm, and you can connect them to form a complete processing pipeline.

## 3. Blocks Window <a href="#id-3-blocks-window" id="id-3-blocks-window"></a>

<figure><img src="/files/YwcZYAsuYU2TUwohMLvA" alt="" height="550"><figcaption><p>Blocks Window</p></figcaption></figure>

The Blocks Window is located on the left side of the interface and contains a collection of blocks that you can use in your workflows.

Everything you need when building your projects resides here.

You can simply drag and drop blocks from the toolbar into the Scenario Area to add them to your workflow.

## 4. Log Window <a href="#id-4-log-window" id="id-4-log-window"></a>

<figure><img src="/files/39I8cXLzOMQeX56GfnPp" alt="" height="550"><figcaption><p>Log Window</p></figcaption></figure>

The Log Window is located at the bottom of the interface and displays real-time logs, warnings, and error messages related to your workflows.


# Scenario Area

Let's explore what we can do with the scenario area.

## Create New Scenario File <a href="#create-new-scenario-file" id="create-new-scenario-file"></a>

Create a new scenario file from **File➡️New**:

<figure><img src="/files/kFNBwYrz2xOgJrcEgSL0" alt=""><figcaption></figcaption></figure>

## Adding Function Blocks <a href="#adding-function-blocks" id="adding-function-blocks"></a>

After creating new project you can drag\&drop function blocks from the right section.

You can drag and drop function blocks with mouse to an emptry scenario.

![](/files/B74IGBtSbjA4OeOBGJt0)

## Connecting Function Blocks <a href="#connecting-function-blocks" id="connecting-function-blocks"></a>

You can connect output and input sockets together:

![](/files/PMtrB29YyEMhOiIIvmRO)

## Search Function Blocks <a href="#search-function-blocks" id="search-function-blocks"></a>

Also you can double click on the empty grid area and search through all function blocks.

![](/files/SWQyJTZwBI6nK2BTIlCh)

You can use keyboard up-down arrow buttons and press enter or click on an item you'd like to add to the scenario.

## Copy-Cut-Paste-Remove <a href="#copy-cut-paste-remove" id="copy-cut-paste-remove"></a>

You can also copy-cut-paste-remove selected function blocks and connections with the contect menu by right clicking:

![Selected items will be highlighted](/files/j2QE4yxbbYK2vtwQrMt4)

or you can use keyboard shotcuts:

* `Ctrl-c`: Copy
* `Ctrl-x`: Cut
* `Ctrl-v`: Paste
* `Del`: Remove

## Add Informative Text <a href="#add-informative-text" id="add-informative-text"></a>

<figure><img src="/files/KRKClTgosh01lFxptr5N" alt="" width="160"><figcaption></figcaption></figure>

By opening the context menu and clicking `Add Text`, you can add informative texts on a scenario:

<figure><img src="/files/XO0sAEX87nQ2jhJF5Cx0" alt="" width="560"><figcaption></figcaption></figure>

## Change Background Style <a href="#change-background-style" id="change-background-style"></a>

You can change the background style:

<figure><img src="/files/A30I4GW0KRHMOO0yQ3P2" alt="" width="560"><figcaption></figcaption></figure>

## Freeze Controls on Blocks <a href="#freeze-controls-on-blocks" id="freeze-controls-on-blocks"></a>

You can freeze controls on a function block to make sure no accidental changes happen with `Ctrl+F`:

<figure><img src="/files/XEoIeDpDQDsj6mdZH9yt" alt="" width="560"><figcaption></figcaption></figure>

That is all! Now, you can have a simple tour on the menubar.


# Menu and Toolbar

## Toolbar <a href="#toolbar" id="toolbar"></a>

![alt text](/files/OfaB1zsx5mZP4IiDp5ZD)

The AugeLab Studio toolbar provides quick access to essential actions and features within the application. Here's a simple overview of the toolbar:

### Actions <a href="#actions" id="actions"></a>

<img src="/files/YR1GdIMrTd2kKZki2AgJ" alt="" data-size="line"> `Run Step` on the current scenario.

<img src="/files/OXdKxS8NTjgLTugdBQ0c" alt="" data-size="line"> `Run` the scenario.

<img src="/files/jGYxKldN8y4qbUEKzQZG" alt="Stop Icon" data-size="line"> `Stop` the scenario.

<img src="/files/bovNV8kdIw3ofpablX9H" alt="" data-size="line"> `Performance Analysis` Toggle performance analysis mode.

<img src="/files/0224COQY9S9jGTHNM3UR" alt="" data-size="line"> `Designer Window` Open the designer window for creating custom blocks.

<img src="/files/Ww3YXLIYFAMdEkOLNkQa" alt="" data-size="line"> `Plugin Window` Open the plugin window to import custom blocks and more.

<img src="/files/MtlAxafwxvXvw5C4X6IB" alt="" data-size="line"> `Maximize` the application window.

<img src="/files/8GCHVZt86Fmyr35vRdmI" alt="" data-size="line"> `Center Blocks View` within the application.

<img src="/files/hbkjemOEkiN6xwv4joV3" alt="" data-size="line"> `Copy` selected blocks within the application.

<img src="/files/WUBf5g9ujatAAeFp4SG2" alt="" data-size="line"> `Cut` selected blocks within the application.

<img src="/files/mk7E0VQCiptXsmFLzFhS" alt="" data-size="line"> `Paste` blocks from the clipboard within the application.

<img src="/files/tjEVIB7Jea2qpxKcIbJj" alt="Toolbar Marketplace Icon" data-size="line"> `Module Downloader` Open the module downloader to install AI modules.

<img src="/files/ozHjsy4RX3PHMLq83Tr6" alt="Toolbar AI Training Icon" data-size="line"> `AI Chat` Open the chat interface.

## Menubar <a href="#menubar" id="menubar"></a>

### File <a href="#file" id="file"></a>

<details>

<summary>File</summary>

* `New` project file\[^1].
* `Example Scenarios`: Check out example scenarios for beginners.
* `Open` the saved project file to load.
* `Open Recent` project files you have used recently from this tab.
* `Save` the current project file over the same file
* `Save as` current project file with a different name and location.
* `Export`scenarios to be used on older versions.
* `Exit` the program and return to the desktop.

</details>

<details>

<summary>Edit</summary>

* `Undo` the last action
* `Redo` last action
* `Cut` selection
* `Copy` selection
* `Paste` selection
* `Delete` selection

</details>

<details>

<summary>View</summary>

* `Close` current scenario
* `Close All` scenarios
* `Next` open scenario
* `Previous` open scenario
* `Center` current blocks in the scenario
* `Toggle Full Screen`

</details>

<details>

<summary>Run</summary>

* `Run One Step` the current scenario
* `Run` the current scenario on loop
* `Stop` the current running scenario
* `Performance Analysis` activation on current scenario
* `Run on Program Start` activates automatic running on the current scenario when you launch AugeLab Studio

</details>

<details>

<summary>Tools</summary>

* Hide/Show `Blocks Toolbar`
* Hide/Show `Log Window`
* Hide/Show `Designer Window`
* `Plugin Window` is used to open the interface that manages custom solutions for you to download
* `Import Package Window` is used to import Python packages to your environment

</details>

<details>

<summary>AI Tools</summary>

* `Module Downloader` is used to install AI modules into your environment
* `Object Detection Training Window` is used to train a special AI model for object detection.
* `Image Annotation Window` is used to prepare your dataset for your AI object detection models.

</details>

<details>

<summary>Help</summary>

* `Demo Projects` contains many example projects for you to try out.
* `About` shows detailed information on the current version of AugeLab Studio
* `What's New` shows the updates made to the latest version.
* `Manual` directs you to this documentation
* `Survey` directs you to our survey to give feedback on AugeLab Studio
* `Reactivate License` is used to change the current license code.

</details>

That's all there is about the menu and toolbar!


# Managing Projects

Managing projects is actually easy with AugeLab Studio.

Described early in the [Menubar-File](/augelab-studio-interface/menubar-toolbar#file) section, you can save-load your project files with `.pmod` extension.

## Handling resource files <a href="#handling-resource-files" id="handling-resource-files"></a>

When haandling resource files, it's best practice to put any resource such as:

* Image files (image, video etc.)
* AI models
* Certificates

in the same folder with the .pmod file.

This way, you'll be able to share your project files and all of external resources without reloading them.

## Exporting Projects to Different Computers <a href="#exporting-projects" id="exporting-projects"></a>

AugeLab Studio .pmod files can be opened on any computer with AugeLab Studio installed.

To keep things simple, organize your resource files (images, weights, files etc.) in the same folder with the .pmod file. Simple folder structure would look like this:

```
  MyProjectFolder
  ├── my_project.pmod
  ├── resources
  │    ├── image1.jpg
  │    ├── model.onnx
  |    └── human_det.weights
```

When you need to move your project to a different computer, just copy the entire folder to the new computer and open the .pmod file with AugeLab Studio. All resource files will be loaded automatically.


# Installing AI and much more

All of features below are optional and some of them are not required for every user. You can always come back here to follow instructions.

<details>

<summary>Installing AI modules</summary>

Refer to [Module Downloader](/augelab-studio-interface/external-features/module-downloader#installing-the-ai-bundle) section to complete the AI modules installation.

</details>

<details>

<summary>Activating Barcode Reader</summary>

You'll need to install [Visual C++ Redistributable Packages for Visual Studio 2013](https://www.microsoft.com/en-US/download/details.aspx?id=40784) for[Barcode Reader](/function-blocks/blocks-reference/detections-shapes/detectors/barcode-reader) to work.

After installation, close any instance of AugeLab Studio and restart your computer.

</details>

<details>

<summary>Activating OCR</summary>

Refer to [Module Downloader](/augelab-studio-interface/external-features/module-downloader#installing-additional-ai-packages) section to complete and download paddleocr package.

</details>

<details>

<summary>Being a plugin developer</summary>

To be able to create and upload plugins. Visual C++ Compiler needs to be installed.

* [Download VS Build Tools](https://studio-desktop-application.s3.eu-central-1.amazonaws.com/vs_BuildTools.exe)

*This step is required only the users who wants to create plugin.*

<figure><img src="/files/A1WFN3IFBhlu3kDGmxyZ" alt=""><figcaption></figcaption></figure>

</details>


# Leverage AI with Module Downloader

Module Downloader is an optional tool accessible via AI Tools ➡️ Module Downloader from the menubar.

### First Look <a href="#first-look" id="first-look"></a>

{% hint style="info" %}
You will need to have a computer with [Nvidia GPU](/introduction/system-requirements) to utilize AI modules.

Without a GPU, AI modules perform very poorly performance-wise.
{% endhint %}

<figure><img src="/files/2qliiWdpOHzSW1VSZPRi" alt=""><figcaption></figcaption></figure>

Module Downloader is a pretty straightforward tool to integrate the power of AI into your environment!

## Installing the AI Bundle <a href="#installing-the-ai-bundle" id="installing-the-ai-bundle"></a>

AI Bundle features a very flexible CNN framework, Object detection algorithms to test and train.

Usage is pretty simple, click on `Download AI Bundle` in the middle of the screen.

Depending on your internet speed, this process may take up to an hour. After the downloading finishes, close AugeLab Studio and restart.

## Installing additional AI Packages <a href="#installing-additional-ai-packages" id="installing-additional-ai-packages"></a>

AI packages under `Integrate 3rd Party Packages` can also be installed by the user.

Please read the description below to check if your setup is eligible or not.


# Demo Projects

Demo projects is available under menubar, in **Help** section:

<figure><img src="/files/TRGXCO9NVImMjOJhZ2aT" alt=""><figcaption></figcaption></figure>

The "Demo Projects" sub-window in AugeLab Studio lets users explore various demo scenarios specific to function blocks.

The Demo Projects sub-window is a valuable tool for learning how to use different blocks and features in AugeLab Studio without programming. Enjoy exploring and learning with the demo projects!


# Creating an Alive Signal

An alive signal, also called a heartbeat, repeatedly changes between `TRUE` and `FALSE` while a scenario is running. An external PLC or device can monitor those changes to confirm that the scenario is still alive.

## 1. Create the pulse

1. Add a [PWM (Pulse Width Modulation)](/function-blocks/blocks-reference/input-output/data-inputs/pwm-pulse-width-modulation) block.
2. Set **Time mode** to **Seconds**.
3. Set **Interval** to the complete cycle and **Up Duration** to the time the output remains `TRUE`.

For example, an interval of `0.3` seconds and an up duration of `0.1` seconds produces `TRUE` for 0.1 seconds and `FALSE` for 0.2 seconds, repeatedly.

## 2. Send the signal to the external device

Connect the PWM block's **Boolean** output to the **Data** or **Value** input of the communication writer used by the device. Connect the writer to its communication client or connection as usual.

* If the writer has an **Enable** input, connect a [Logic Input](/function-blocks/blocks-reference/input-output/data-inputs/logic-input) set to `TRUE`.
* For a TCP writer such as **TCPIPWrite**, set **Send Policy** to **Always** so both signal states are sent. The example below uses this configuration.
* For [Siemens S7 Write](/function-blocks/blocks-reference/input-output/communication/siemens-s7-write), connect **Siemens S7 Connect** to **S7 Client**, set **DB Data Type** to `Boolean`, and configure the required DB number, byte address, and bit position.
* For [Modbus Write](/function-blocks/blocks-reference/input-output/communication/modbus-write), connect **Modbus Connect** to **Modbus Client**, choose **Coil**, and set the target address.

<figure><img src="/files/dQXN56hcTmNJxuDgI9ML" alt="PWM Boolean output connected to the Data input of TCPIPWrite, with a TRUE Logic Input connected to Enable."><figcaption><p>Example heartbeat sent through TCPIPWrite</p></figcaption></figure>

## 3. Configure the receiver

The external device should monitor for a change of state, rather than the current value alone. With the example above, a timeout of about one second allows for normal scheduling and network jitter while still detecting a stopped scenario quickly.

* Use a dedicated PLC bit or tag; do not share it with process-control logic.
* Choose a slower pulse if the device, PLC scan time, or network cannot reliably observe short transitions.
* If the writer provides **Data on Stop**, set it to `FALSE` when the receiver needs a known final state.

The external device raises its timeout alarm when it no longer receives state changes. The signal stops when the scenario stops evaluating the PWM and communication blocks.

## 4. Test the signal

Before connecting to production equipment, use [Scope](/function-blocks/blocks-reference/input-output/outputs-exports/scope) to confirm the expected `TRUE`/`FALSE` cycle. Then verify at the receiver that it sees both states and raises an alarm after the configured timeout.


# Circumference Measurement

You can measure an object's several features such as width, height, and circumference using AugeLab Studio's native function blocks.

In this example, will separate an object from its background and measure its area, width, and height.

First, Use **Load Image** block and load **paper.jpg** from the example images folder.

With [**HSV Filter**](/function-blocks/blocks-reference/image-transformations/color-filters/hsv-filter) and adjusting Hue, Saturation, and Values we'll try to separate the calculator from the background:

<figure><img src="/files/TEGU6u0gkrVmG7LJMipR" alt=""><figcaption><p>Click to englarge</p></figcaption></figure>

You may run the scenario one step everytime you adjust the sliders in **HSV Filter** block to see differen outputs. To understand what does HSV mean, refer to [HSV Filter](/function-blocks/blocks-reference/image-transformations/color-filters/hsv-filter) documentation.

{% hint style="info" %}
You may use [Blur](/function-blocks/blocks-reference/image-transformations/color-filters/blur) or other pre-processing blocks to ease out random noise from images. However, it's always better to keep preprocessing to minimum when dealing with measurements.
{% endhint %}

Now that have successfully separated the outline of object from the background, we can use [**Edge Filter**](/function-blocks/blocks-reference/image-transformations/color-filters/edge-filter) and [**Find Contour**](/function-blocks/blocks-reference/detections-shapes/shape-analysis/find-contour) to extract the shape of our object. Add these blocks to the scenario accordingly and tweak with slider values:

<figure><img src="/files/uaDlftpDoWS4pb8fl8DC" alt=""><figcaption></figcaption></figure>

Contours are polygons that consist of several points on the 2D image space. Using contours, you can calculate their circumference, center point, angle, etc. However, the contours themselves do not contain information about their width and height since they are polygons with an unknown number of edges.

Calculating width and height would require [Minimum Rotated Rectagle](/function-blocks/blocks-reference/detections-shapes/shape-analysis/minimum-rotated-rectangle) block. Combining this with **Find Contour** we'll be able to calculate width, height, and area:

<figure><img src="/files/ixIuKay8KNrfIrRl89M5" alt=""><figcaption></figcaption></figure>

Calculated width, height, and area do not represent real-life units. They represent the number of pixels. In order to convert them to metric units, you'll need a constant to convert from pixel to unit length.

There you have it! This tutorial have shown you how to calculate the area, width, and height of an object by the separation method. You may check other [shape analysis](https://github.com/AugelabTech/AugeLab-Studio-Gitbook-Docs/blob/main/english/blocks/blocks-reference/imgproc2/shape_analysis/README.md) methods to work with different shapes.


# Object Counting

Counting objects is a widely encountered problem in the computer vision field. This tutorial will teach you how to count circular objects in a given area using conventional computer vision algorithms.

The example image is already provided in AugeLab Studio in the example images folder as **coins2.jpg** file.

Create [Load Image](/function-blocks/blocks-reference/input-output/image-inputs/load-image) block in an empty scenario with the example image shown below:

<figure><img src="/files/D01YdjY0d3i7sxRTIGSj" alt=""><figcaption></figcaption></figure>

As a first step, we'll need to separate the coins from the background using the [Image Threshold](/function-blocks/blocks-reference/image-transformations/color-filters/image-threshold) block. This can also be done with [HSV Filter](/function-blocks/blocks-reference/image-transformations/color-filters/hsv-filter) or [RGB Mask](/function-blocks/blocks-reference/image-transformations/color-filters/rgb-mask) but separating the color areas and acquiring a binary image will suffice. Create the logic below:

<figure><img src="/files/vcZPBHVIdB0BkGbX4ib9" alt=""><figcaption></figcaption></figure>

Since we'll be using the [Find Contour](/function-blocks/blocks-reference/detections-shapes/shape-analysis/find-contour) block to count how many distinct white area exist, we'll be selecting **THRESH\_BINARY\_INV** option to filter the image and adjust the slider to filter the background.

However, you may see that white areas are not separated from each other perfectly. Using the Find Contour block would yield the wrong result:

<figure><img src="/files/n2273E3EvTczwCwjy15c" alt=""><figcaption></figcaption></figure>

As you see, there aren't 14 block coins in the provided image. We'll need an algorithm to separate or shrink the white areas. For this, we'll be using the [Distance Transformation](/function-blocks/blocks-reference/image-transformations/transformation-filters/distance-transformation) block:

<figure><img src="/files/pm7Os1HJ87EZjtFQNRM0" alt=""><figcaption></figcaption></figure>

Distance transformation calculates how far away each pixel from white color density. Using [Image Threshold](/function-blocks/blocks-reference/image-transformations/color-filters/image-threshold) again will yield distinct white areas for each one:

<figure><img src="/files/WcgGQxRliL5Mfo3jR1KK" alt=""><figcaption></figcaption></figure>

Now, using the find contour block should yield how many coins we have the in reference image:

<figure><img src="/files/FywCj6nstu6sF5S7UnOB" alt=""><figcaption></figcaption></figure>

That's it! You now know how to count each object in a given area with AugeLab Studio!


# Tile Width Measurement

### Measuring Tile Width <a href="#measuring-tile-width" id="measuring-tile-width"></a>

Measurements can be challenging based on the context of the project. AugeLab Studio's ready to use function blocks allows easy and fast measurements on a such challenging subject.

This example will show you how to measure the width (effectively distance) of a wooden tile.

First. Use [Load Image](/function-blocks/blocks-reference/input-output/image-inputs/load-image) function block and load **wood.jpg** from example images folder provided by AugeLab Studio.

<figure><img src="/files/Ww6LZCeDX1ODwgKSXtbU" alt=""><figcaption></figcaption></figure>

Our aim is to measure each tiles width and print them out. Next, we'll need to pre-process this image to lower noise that can be introduced by the camera or environment. For such cases, [Blur](/function-blocks/blocks-reference/image-transformations/color-filters/blur) function block is a very good match with **Median Blur** option. Go ahead and create the logic below:

<figure><img src="/files/dEOPgNSYViz48eMP8ag9" alt=""><figcaption></figcaption></figure>

We are keeping *Kernel Size* at 3 to keep pre-processing to minimum, since any kind of pre-processing can affect the final outcome during measurements. Next, we'll be using [Histogram on Line](/function-blocks/blocks-reference/image-transformations/analysis/histogram-on-line) function block.

**Histogram on Line** function block automatically detects sharp edges, stores their locations as points by the given threshold and position values. By creating the logic below:

<figure><img src="/files/zEBQZs4sHiAHP3Q2Bpej" alt=""><figcaption></figcaption></figure>

We are telling **Histogram on Line** block to calculate edges at 100th horizontal pixel position, and look for edges that exceed values over 100 (which can be between 0-255).

The horizontal location of detection line (second socket) is most important, since other factors can easily cause false detections. Lets provide 350 and see what happens in such scenario:

<figure><img src="/files/jfbkzj0IGYPV5gJ293Dh" alt=""><figcaption></figcaption></figure>

Choosing a horizontal line with more disturbances has caused false detections. Therefore, it is always recommended to choose a line that is minimally affected by noise and environment.

Since we have successfully calculated the edge locations of each tile, we can calculate the width of first tile by introducing several blocks.

First, we'll be using *Peak Mean Locations* output by the **Histogram on Line** block. This output consist of a [list](/function-blocks/sockets#purple-position)[ of positions](/function-blocks/sockets#purple-position):

<figure><img src="/files/ZapLYPCtyXmwEZbeh9vh" alt=""><figcaption></figcaption></figure>

Each position contains horizontal and vertical positions (x, y) of peak locations. We can calculate the distance between each point by either using [Demux](/function-blocks/blocks-reference/data-logic/logic/demux) block, or using [Measure Position Distance](/function-blocks/blocks-reference/image-transformations/analysis/measure-position-distance) function block.

Use [List Operations](/function-blocks/blocks-reference/data-logic/data-operations/list-operations) block and choose **get** from the drop-down menu. Follow the logic below:

<figure><img src="/files/6Mfky8EFH4czZHYaQY9R" alt=""><figcaption></figcaption></figure>

The logic above selects the first element by providing zero (0) and the second element by providing one (1). Using [Measure Position Distance](/function-blocks/blocks-reference/image-transformations/analysis/measure-position-distance); x-y, and Euclidean distances are calculated and presented.

You may have noticed that y distance is zero, due to all points being at the same horizontal line.

That's it! You successfully calculated the width of the first tile.

### Calculating all tile widths <a href="#calculating-all-tile-widths" id="calculating-all-tile-widths"></a>

Now, let's calculate the other widths by using list and batch operations.

By using the same **List Operations** function block, we'll use the *pop* option from the dropbox. Pop option removes the item at given index from a list. Let's create the logic below again:

<figure><img src="/files/CluTLBx0O8UAxIaB3Qzf" alt=""><figcaption></figcaption></figure>

**List Operations** block first copies the provided list and then calculates the desired outcome. We created a list of peak locations without the first element, and then created another list by only removing the last one.

Using them with [Batch Processing](/function-blocks/blocks-reference/data-logic/flow-control/batch-processing) with [Data Type Converter](/function-blocks/blocks-reference/data-logic/data-operations/data-type-converter), we can calculate positional difference between each peak points:

<figure><img src="/files/VEkAZ3hLUdvvzwxiV8Y5" alt=""><figcaption></figcaption></figure>

It's important to use [Data Type Converter](/function-blocks/blocks-reference/data-logic/data-operations/data-type-converter) when dealing with Batch Processing. If a function block receives a batch, it runs in batch mode. Use *Batch2List* option from its menu to revert to normal operation mode.

That's all! You've calculated the width of all tiles!


# Human Detection

Human detection has been a very hot topic in both computer vision and public media and falls under the same category as [Object Detection](/example-projects/object-detection). This short documentation will help you detect the positions on humans and count them entering an area.

{% hint style="info" %}
You'll need to install AI modules and have a computer with GPU to complete this tutorial. Please refer to the [installation guide](/augelab-studio-interface/external-features/module-downloader#installing-the-ai-bundle).
{% endhint %}

A simple human detection case can be tested with a simple scenario like below:

<figure><img src="/files/i1UTHajk3UeymGtpQIqX" alt=""><figcaption></figcaption></figure>

For our case, will be using [Video](/function-blocks/blocks-reference/input-output/image-inputs/video), [Object Detection](/function-blocks/blocks-reference/ai-blocks/object-detection) and [Show Image ](/function-blocks/blocks-reference/input-output/outputs-exports/show-image)blocks in AugeLab Studio. You can drag-drop these blocks from Blocks section or double click on the empty scenario and type their name.

Go ahead and click on **Select Video File** on Video block and choose **footage.mp4** on *example images* folder.

Disable *Real-time* on **Video** because we would like to process each frame with **Object Detection**.

Choose *Human* on *Object Detection* block in detection class selection box. Slide the confidence threshold to %50.

Press on F5 or Window->Run->Run One Step and you should encounter this scene:

<figure><img src="/files/yQ6sZDRq7By5cYUkRx71" alt=""><figcaption></figcaption></figure>

You may see there are several human detections on our video file. However, we would like to count how many people enter in or exit out of our camera perpective. To achieve that, we'll be using **Check Area** block:

<figure><img src="/files/2v0HtBGjVxsinfv9soDz" alt=""><figcaption></figcaption></figure>

Connect sockets as provided in the figure above, run the scenario for one step and draw the detection area as well in **Check Area** block.

Running the current scenario will count how many objects are in the drawn rectangle:

<figure><img src="/files/t3icvhYvW6RChXlS6i26" alt=""><figcaption></figcaption></figure>

However, we will only be counting objects in this area at current time, we won't be able to hold information on how many people have passed the area. To calculate that, we need to create the logic below:

<figure><img src="/files/6brh4SCdOE0CNmE8FLCC" alt=""><figcaption><p>Click to enlarge</p></figcaption></figure>

The logic above subtracts the total number of detected objects from the previous state using **Delay Step** block. If there is a difference greater than one, it adds up to it and saves it with **Counter** block.

There you have it! Now, run this scenario with **Ctrl+F5** or **Window->Run->Run** and you'll be able to count the number of people passing through a certain area:

<figure><img src="/files/bhwb4IpvYBa9SNlKNCHs" alt=""><figcaption></figcaption></figure>


# Object Detection

{% hint style="info" %}
You'll need to install AI modules and have a computer with GPU to complete this tutorial. Please refer to the [installation guide](/augelab-studio-interface/external-features/module-downloader#installing-the-ai-bundle) for further instructions.
{% endhint %}

Object detection has been a very hot topic in both computer vision and public media. This practice is widely applied to many different industries and has many more potential application areas.

This tutorial will show you how to create a simple collision warning system on a public bus.

### Footage <a href="#footage" id="footage"></a>

As for any scenario, we'll need front-side footage of a public bus. Go ahead and download the dash-cam video:

{% file src="/files/G88Zi6gerTPJ07eRUden" %}

Using [Video](/function-blocks/blocks-reference/input-output/image-inputs/video) block, we'll be reading the dash cam results. We'll also need [Object Detection](/function-blocks/blocks-reference/ai-blocks/object-detection) block to detect any human, bike or cars that we might come in contact with.

Go ahead and create the scenario below:

<figure><img src="/files/tOcGBALR8pHwGwFFyVbE" alt=""><figcaption></figcaption></figure>

Since our collision detection system should only check if there is an object standing in front of our bus, we'll have to use the [Check Area](/function-blocks/blocks-reference/detections-shapes/roi-processing/check-area) block and select the collision warning area:

<figure><img src="/files/B74x3FNEExvDHnPWrIA1" alt=""><figcaption></figcaption></figure>

[Check Area](/function-blocks/blocks-reference/detections-shapes/roi-processing/check-area) block will allow us to assess if there are any objects in our reference bounding box area. Use [Not](/function-blocks/blocks-reference/data-logic/logic/not) and [Led Output](/function-blocks/blocks-reference/input-output/outputs-exports/led-output) blocks to indicate if our collision system works as expected or not:

<figure><img src="/files/QXSVQqIPiD1t64FhtnnO" alt=""><figcaption></figcaption></figure>


# AI Assistant | HMI, Workflows and Custom Blocks

<figure><img src="/files/ESTocvz5xjG7tK7LC6tQ" alt="" width="250"><figcaption><p>AI Agent</p></figcaption></figure>

AugeLab AI is your in-app expert for features, troubleshooting, and workflow design. For the best results, always provide **Context** (Goal, Setup, Action, and Expected Result).

<details>

<summary>🛠️ Building Workflows &#x26; Scenarios</summary>

The AI builds better logic when it knows your environment and constraints.

* **Details to Include:** Input source (Camera/Folder), Output goal (Measurement/Detection), and Constraints (Target FPS/Accuracy).
* **Example Prompt:** > *Goal: Detect missing silver parts on a black conveyor belt moving at 0.2m/s. Setup: Windows 11, NVIDIA GPU, Basler camera. Expected: Recommended node sequence and tuning tips.*

<figure><img src="/files/ESTocvz5xjG7tK7LC6tQ" alt="AI Agent Interface" width="250"><figcaption><p>AI Agent Interface</p></figcaption></figure>

</details>

<details>

<summary>🧩 Designing Custom Blocks</summary>

Describe custom blocks like a "mini product spec" for the AI to generate the correct Python structure.

* **Details to Include:** I/O types (Image, Bool, etc.), user-adjustable parameters, and pass/fail logic.
* **Example Prompt:** > *Goal: Create a block that checks if a part is in a safe window. Input: Detection (x,y,w,h). Output: PASS/FAIL for HMI. Need parameters for tolerance and debounce.*

<figure><img src="/files/L9LDixmp9KKzP5u3J5KL" alt="Custom Block Creation" width="250"><figcaption><p>Custom Block Logic</p></figcaption></figure>

</details>

<details>

<summary>🖥️ Creating HMI Applications</summary>

Focus on **who** uses the screen and **what** decisions they need to make.

* **Details to Include:** Target device (Touchscreen/PC), essential glanceable data (Counters/Status), and required actions (Start/Stop/Reset).
* **Example Prompt:** > *Goal: Simple operator HMI for PASS/FAIL station. Needs a large status indicator and a reset button for alarms.*

</details>

<details>

<summary>🐞 Troubleshooting &#x26; Performance</summary>

To fix errors or "lag," the AI needs evidence.

* **For Bugs:** Provide exact error text, steps to reproduce, and logs.
* **For Performance:** Note which action is slow (e.g., "Inference taking 500ms") and your GPU usage.
* **Example Prompt:** > *Goal: Fix crash on Chat panel. Setup: Windows 11. Action: Clicked Chat menu. Result: App closed immediately. Log: \[Attached].*

<figure><img src="/files/pbxhkv9adlqwC0rzoLaL" alt="Troubleshooting with logs" width="250"><figcaption><p>Debugging with AI Agent</p></figcaption></figure>

</details>

<details>

<summary>📸 Effective Screenshots &#x26; Privacy</summary>

* **Capture the whole panel:** Show the selected node and the settings panel together.
* **Highlight:** Use arrows or circles to point at the specific error or missing button.
* **🔒 Privacy First:** Never share license keys, passwords, or sensitive customer data. Redact images before uploading.

</details>

***

### 💡 If the Assistant gets it wrong:

Don’t start a new chat. Provide a specific correction:

> *"That's not quite right. I'm specifically using the \[Panel Name] and the issue is \[X]. Please avoid using \[Node Y]."*


# Widget to Socket Utility

Widget to Socket turns a node controls into an input socket. Use it when a value that is normally typed, selected, or dragged inside a node must come from another block instead.

Example uses:

* Drive a threshold from a calculation.
* Let an HMI or manual input control a node parameter.
* Feed settings from a PLC, barcode reader, recipe file, or headless script.
* Test several parameter values without opening the node and editing the widget each time.

This page walks through one small scenario. You will convert a widget into a socket, connect a dynamic value, run the scenario, then convert it back.

## What You Will Build

You will create a range value from two number inputs and feed it into a converted widget socket.

<figure><img src="/files/QG1YrEZE1lgxqTJqMqM4" alt="Scenario before converting a widget to a socket"><figcaption><p>Start with a node that still uses its normal on-node widget.</p></figcaption></figure>

## Step 1: Add Example Blocks

Create a new scenario and add these blocks:

1. Add two `Number Input` blocks.
2. Add one `Mux` block.
3. Add one `Number Range` block.
4. Add one output or debug block so you can inspect the result.

Set the first `Number Input` to the lower value, for example `10`.

Set the second `Number Input` to the upper value, for example `80`.

Connect both number inputs into `Mux`.

{% hint style="info" %}
This example uses `Number Range` because the range slider is easy to recognize. The same workflow applies to other node widgets that show the API conversion action.
{% endhint %}

## Step 2: Open the Widget Menu

Right-click the widget you want to control from another block.

On the `Number Range` block, right-click the `Range Slider` widget.

<figure><img src="/files/pyaFH8YEisbXyG1IEjDr" alt="Widget context menu with API conversion action"><figcaption><p>Right-click the widget and choose the API conversion action.</p></figcaption></figure>

Choose:

```
[API] Convert to socket
```

## Step 3: Confirm the New Socket

After conversion, the widget is hidden and a new input socket appears on the node.

<figure><img src="/files/UKESfENzwCuDQ5HvlmNa" alt="New input socket created from the converted widget"><figcaption><p>The widget setting is now available as an input socket.</p></figcaption></figure>

The socket name matches the converted widget name. This makes it easier to understand which parameter you are controlling.

## Step 4: Connect the Dynamic Value

Connect the `Mux` output to the new converted socket on `Number Range`.

<figure><img src="/files/VBDMtuAjeW5mgJTyXX4r" alt="Mux output connected to the converted widget socket"><figcaption><p>The node parameter now comes from another part of the scenario.</p></figcaption></figure>

Run the scenario. The `Number Range` block now reads the incoming value instead of the hidden slider widget.

Change either `Number Input` value and run again. The range updates through the socket connection.

## Step 5: Convert Back If Needed

If you no longer want external control, right-click the converted socket and choose:

```
Convert to widget
```

<figure><img src="/files/jqk8yA37QsX4LgLcT6fD" alt="Converted socket menu with Convert to widget action"><figcaption><p>Converted sockets can be restored to their original widget form.</p></figcaption></figure>

The input socket is removed and the original widget appears again.

## When to Use This

Use Widget to Socket when a parameter should be controlled by scenario logic.

Good cases:

* Recipe-driven thresholds.
* Operator controls from an HMI.
* Headless scenarios where values come from a script or file.
* Automated test scenarios that sweep through many parameter values.
* Shared parameter values used by several nodes.

Avoid it when the value is always constant. Keeping a normal widget is simpler when no other block needs to control it.

## Troubleshooting

If `[API] Convert to socket` is missing, that widget is not convertible.

If the node result does not change, check that the connected block is producing a value every run.

If the converted socket rejects a connection, the source value type may not match what the widget expects. Use a converter, `Mux`, `Demux`, or a matching input block.

If the scenario becomes harder to read, rename nearby blocks so the parameter source is obvious.


# Annotate Data for Object Detection

Data collection and annotation is the foundation in training an object detection model.

<figure><img src="/files/Y4THPuhXTgsQW8dtz7rt" alt="Annotate Data for Object Detection" width="400"><figcaption><p>Annotate Data for Object Detection</p></figcaption></figure>

This section of the guide will walk you through data collection and annotation using the Image Annotation Window in AugeLab Studio.


# Dataset Collection

The fastest way to build a high-performance AI model is to **capture data on purpose**. This page covers how to collect high-quality images and videos using AugeLab Studio's native tools.

{% hint style="info" %}
You may skip this section if you already have a folder of images/videos ready for annotation.
{% endhint %}

***

## Planning Your Dataset

It's crucial to plan your dataset before collection. A well-structured dataset leads to better model performance.

### 📊 How Much Data Do You Need?

The number of images required depends on how much the environment changes. Use this table as a starting point for your collection goal.

| Project Type        | Environment                                     | Recommended Images per class\* |
| ------------------- | ----------------------------------------------- | ------------------------------ |
| **Simple**          | Controlled lighting, fixed camera, 1-2 classes. | **50 - 150**                   |
| **Industrial**      | Factory floor, changing shifts, conveyor belt.  | **200 - 500**                  |
| **Complex**         | Variable lighting, many classes, moving camera. | **1,000+**                     |
| **Complex Outdoor** | Outdoor scenes with weather changes.            | **2,000+**                     |
| **Rare Event**      | Detecting occasional defects or leaks.          | **50 Target / 100 Empty**      |

> \*Images per class refers to the number of annotated instances of each object category, not just total images.

{% hint style="info" %}
For best results, aim for **diversity** in angles, distances, and lighting within your dataset.
{% endhint %}

{% hint style="warning" %}
Number of classes should be consistent across the dataset. If not, **augmentation** can help balance classes later.
{% endhint %}

***

### 🏗️ Define the boundaries:

Write these down before taking the first photo to ensure your dataset is **Representative** and **Consistent**.

1. **Class List**: What specific objects are you detecting?
2. **Camera Specs**: What is the final mounting angle, distance, and Field of View (FoV)? Single or multiple cameras?
3. **Variations**: Will there be shifts in lighting (glare/shadows) or background clutter?
4. **Negatives**: What does an "empty" scene look like?
5. **Scope**: What objects should the model intentionally ignore?

***

## Camera Configuration

Whether using a USB camera, IP camera, or industrial camera, ensure the following settings are optimized before collection:

* **Resolution**: Aim for 480p to 720p (640x480 is a common standard). Higher resolutions can be downscaled later.
* **Frame Rate**: 15-30 FPS is sufficient for most object detection tasks.
* **Focus**: Set to manual focus to avoid shifts during collection.
* **Exposure**: Use manual exposure settings to maintain consistent lighting.
* **Save Settings**: Save your camera settings profile, most cameras allow saving presets, so the settings remain consistent across sessions.

## Dataset Collection

You can collect images and videos for your object detection dataset directly within AugeLab Studio using built-in tools. This ensures compatibility and streamlines the annotation process.

> Another option is to download public datasets or use external cameras/software, but this may require additional formatting steps.

### Capture Inside AugeLab Studio

Using the Studio environment allows you to use triggers (buttons, PLC signals, or timers) to automate your collection.

### 1. Start from the Example Project

AugeLab ships with a pre-configured template for this exact task.

* **Path**: `File` → `Example Projects` (or "Example Scenarios")
* **Project**: **"Data Collection for AI Training"**

### 📸 Single Images: The `Image Write` Block

Use this for high-quality static frames. It is best for "same scene, many positions."

| Input/Setting      | Logic                                                                 |
| ------------------ | --------------------------------------------------------------------- |
| **Folder Path**    | Where images are stored.                                              |
| **Save (Trigger)** | Set to `True` to capture a frame. Pair this with a button or a timer. |
| **Compress Image** | **Checked** = `.jpg` (Smaller)                                        |

### 🎥 Continuous Motion: The `Record Video` Block

Best for conveyor belts or fast-moving inspections where you intend to extract frames later.

| Input/Setting              | Logic                                   |
| -------------------------- | --------------------------------------- |
| **Video Quality**          | **Compressed** = `.mp4`                 |
| **Trigger Mode: Spacebar** | Press Space to Start/Stop.              |
| **Trigger Mode: Once**     | `Record=True` toggles recording on/off. |

> Plan recordings as short, focused clips (10–60s) rather than one massive file. This makes frame extraction much easier.

***

## 📉 Collecting Background (Negative) Images

A robust model needs to know what *not* to detect. You must capture "Empty" scenes on purpose.

* **What to capture**: Empty conveyors, empty workstations, or common non-target objects (fixtures, tools).
* **Empty**: An annotation file exists, but has no boxes.
* **Excluded**: No annotation file exists.

***

## Public Datasets

If you need to supplement your own data, consider these public datasets:

* [COCO Dataset](https://cocodataset.org/#home): Large-scale object detection, segmentation, and captioning dataset.
* [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/): Standard dataset for object detection and segmentation.
* [Open Images Dataset](https://storage.googleapis.com/openimages/web/index.html): A dataset with \~9 million images annotated with image-level labels and bounding boxes.
* [ImageNet](http://www.image-net.org/): Large visual database designed for use in visual object recognition research.
* [Kaggle Datasets](https://www.kaggle.com/datasets): Various datasets for machine learning, including object detection.

## 📂 Folder Structure & Preparation

AugeLab Studio loads datasets by folder. Ensure your structure looks like this:

```
my_dataset/
  ├── 000001.jpg
  ├── 000002.jpg
  ├── background_01.png
  └── classes.names  <-- (Optional, will be created during annotation)
```

***

## 🏁 Capture Checklist

| Check          | Requirement                                                       |
| -------------- | ----------------------------------------------------------------- |
| **Quality**    | Avoid heavy motion blur or over-exposure where edges disappear.   |
| **Coverage**   | Capture objects in the center, corners, and edges of the frame.   |
| **Scale**      | Match the real-world distance from the camera to the object.      |
| **Clutter**    | Include the messy backgrounds the camera will actually see.       |
| **Resolution** | Most AI models work best between 480p and 720p (640x480 average). |


# Annotation Window Basics

## Annotate Data for Object Detection <a href="#annotate-data-for-object-detection" id="annotate-data-for-object-detection"></a>

### First Look <a href="#first-look" id="first-look"></a>

{% hint style="info" %}
You will need a computer with a compatible [Nvidia GPU](/introduction/system-requirements), AugeLab Studio AI/GPU support enabled, and the required bundle installed through [Module Downloader Window](/augelab-studio-interface/external-features/module-downloader#installing-the-ai-bundle).
{% endhint %}

<figure><img src="/files/20o4pNt6RaxyEi4MxPxP" alt=""><figcaption></figcaption></figure>

The AugeLab Studio Image Annotation Window allows users to annotate images by drawing bounding boxes around objects of interest and associating them with specific classes.

### Getting Started <a href="#getting-started" id="getting-started"></a>

To open the Image Annotation Window, navigate to the top menu and click on `AI Tools` ➡️ `Image Annotation`.

For image annotation, you need two things:

1. `.class` file
2. Dataset

### Class File <a href="#class-file" id="class-file"></a>

To label your data, first you need a `classes.names` file, which is a standard text file with *.names* extension. A normal class file looks like below:

```
Human
Dog
Cat
Cup

```

If you do not have such file, you can create your own using the `Classes` section:

<figure><img src="/files/Zos4XebocBdBpveQaZzL" alt="" width="250"><figcaption></figcaption></figure>

To create your own classes file:

1. Type a class name
2. Click on `+` and add your classes.
3. Click on `Save Classes`(third button) and you are ready to pick a folder.

You may also click on `-` to remove any unwanted classes.

### Load Image Folder <a href="#load-image-folder" id="load-image-folder"></a>

{% hint style="warning" %}
Make sure path to your dataset do not contain any non-english characters.
{% endhint %}

Click on `Open Folder` at the top of the screen and choose the folder that contains all of your images:

<figure><img src="/files/Z8bovWbFyGJSN63ArcPg" alt=""><figcaption></figcaption></figure>

After clicking on `Open Folder`, a dialog will appear asking you to choose a folder and a class file:

<figure><img src="/files/z8NZM4qGAmpf1dTg9bEc" alt=""><figcaption></figcaption></figure>

Select Image from List: After loading the image folder, a list of available images will be displayed. Click on an image to select it for annotation.

<figure><img src="/files/FYUaaLimB5CJCegJiSeV" alt="" width="563"><figcaption></figcaption></figure>

### Annotating Images <a href="#annotating-images" id="annotating-images"></a>

Annotating images are pretty simple. Click on the top left of the object you'd like to detect, drag the mouse and release it when you are done!

![Annotation Window](/files/qbuCJrLZedeE11Hl15q3)

Bouding boxes should tightly fit around the object of interest without including too much background. This helps the model learn to focus on the relevant features of the object.

{% tabs %}
{% tab title="Bad Annotation" %}

<figure><img src="/files/j7a0DbPCsZTrcWeXWxBS" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Good Annotation" %}

<figure><img src="/files/rJG9U0UKPQEfo3LSHaNO" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

### Using the Dataset Panel <a href="#using-the-dataset-panel" id="using-the-dataset-panel"></a>

<figure><img src="/files/nEHcs2GiZmjbtk2qX1eB" alt=""><figcaption></figcaption></figure>

Dataset features several functionalities:

1. `Filter` function that allows you to filter several image classes:
   * `All` All images with and without annotation
   * `Annotated` images have annotations with them.
   * `Empty` images do no contain annotations, but included in the training. This means objects that are not annotated may negatively impact training.
   * `Excluded` images do no have an annotation file. This means they do not affect training whatsoever.
2. `Search` functionality will allow you to filter images with their names.

### Annotating Videos <a href="#annotating-videos" id="annotating-videos"></a>

You can also annotate video files using the **Video** mode on the top side of the window:

![](/files/hsguJ79gkLLEHxc8cbM9)

Changing video mode will ask you for a file path. Choosing the video will allow you to annotate a video just like a folder!

### Tools <a href="#tools" id="tools"></a>

There are several tools inside the Annotation Tool to help you during your dataset preparation:

#### Class Frequency Analysis <a href="#class-frequency-analysis" id="class-frequency-analysis"></a>

Clicking on class frequency analysis will analyze and show you how many classes exist in your dataset.

This is useful to check if you have a balanced dataset or not.

#### Augment Dataset <a href="#augment-dataset" id="augment-dataset"></a>

AugeLab Studio automatically applies dataset augmentation. Augmentation is the process of artificially creating similar data.

This subject is detaily covered in [**Augmenting Your Dataset**](/key-features/annotate-data-for-object-detection/augment-dataset) page.

## 🛠️ Troubleshooting AI Vision

If your AI models aren't behaving as expected, use these quick-fix toggles to tune your performance.

<details>

<summary>🚫 "It annotates nothing" (Zero Detections)</summary>

When the AI is being too "shy" to label anything, it's usually a threshold or description issue.

* **Lower Confidence:** Drop the **Confidence Threshold** slightly (e.g., ).
* **Text Sensitivity:** For Grounding DINO, lower the **Text Threshold** to be less strict about word matching.
* **Be Specific:** Instead of "part," try "silver metal bolt" or "red plastic cap." Descriptions should be visual.
* **Check Lists:** Verify that your class list is actually loaded in the node settings and isn't empty.

</details>

<details>

<summary>📦 "Too many wrong boxes" (Ghost Detections)</summary>

If your screen is cluttered with false positives, you need to tighten the "strictness" of the model.

* **Raise Confidence:** Increase the **Confidence Threshold** to filter out low-certainty guesses.
* **Text Strictness:** Increase the **Text Threshold** to force a closer match between the image and your prompt.
* **Remove Ambiguity:** Avoid broad prompts like "object" or "item." If the AI is labeling shadows as "parts," specifically describe the part's unique colors or textures.

</details>

<details>

<summary>❓ "YOLO model doesn't detect my class"</summary>

Standard YOLO models are pre-trained on specific datasets.

* **COCO Standard:** Basic YOLO models only recognize the 80 COCO categories. Your labels must match exactly (e.g., `person`, `cell phone`, `chair`, `bottle`).
* **Custom Needs:** If you need to detect something specific (like a "scratched circuit board"), switch to a **Text-Prompt** model (like Grounding DINO) or train a **Custom YOLO** model.

</details>

<details>

<summary>🐌 "Processing is slow or laggy"</summary>

Vision models are computationally expensive.

* **First-Run Delay:** It is normal for the first run to be slow while models download and initialize in memory.
* **Model Size:** Grounding DINO Base and OWLv2 Large are high-accuracy but "heavy." Try a "Tiny" or "Small" variant for faster speeds.
* **Hardware:** Ensure AugeLab is utilizing your **GPU**. Running large AI models on a CPU will result in significant latency.

</details>

***

#### 💡 Still stuck?

Try the **AI Assistant** in AugeLab Studio. Describe your specific camera view and what the boxes currently look like; it can often suggest the exact decimal value for your thresholds.

Would you like me to create a **"Threshold Cheat Sheet"** table that explains exactly what Confidence vs. Text thresholds do?]\(./augment-dataset.md).

{% hint style="warning" %}
Augmentation process should be done after finishing annotation
{% endhint %}

{% hint style="warning" %}
Augmentation process may increase the disk size of your dataset up to 10 times.
{% endhint %}

#### Preprocess Image <a href="#preprocess-image" id="preprocess-image"></a>

Preprocess Image tool allows you to change the contrast, brightness and gamma of images that are shown in the window. This feature comes in handy when dealing with very dark or too bright images.

![](/files/1M5pi3MTHxSFArecw3yf)

#### Change Class Id <a href="#change-class-id" id="change-class-id"></a>

Change Class Id tool will allow you to change the all annotated class instances to a different class.

This tool comes in hand when merging two different datasets.

![](/files/tjSltRmZJySE0d3mgLXJ)

#### Shortcuts and Help <a href="#shortcuts-and-help" id="shortcuts-and-help"></a>

<details>

<summary>For shorcuts and help, you can click on the `Help` button at the top menu.</summary>

* `D`: Show next image or frame.
* `A`: Show previous image or frame.
* `Shift + D`: Move forward by 10 images/frames.
* `Shift + A`: Move backward by 10 images/frames.
* `W`: Decrement class selection.
* `S`: Increment class selection.
* `Shift + W`: Decrement class selection by 3.
* `Shift + S`: Increment class selection by 3.
* `X`: Remove the last bounding box annotation.
* `Shift + C`: Clear all annotations.
* `O`: Add an empty annotation file or clear annotations.
* `P`: Remove annotations and clear the file.
* `M`: Move or exclude image to another folder (Folder Mode only).
* `Shift + Delete`: Remove image and annotation from computer (Folder Mode only).

</details>

#### Training With Custom AI Object Detection Model <a href="#training-with-custom-ai-object-detection-model" id="training-with-custom-ai-object-detection-model"></a>

To train a custom object detection model, please refer to [**Object Detection Train**](/key-features/train-custom-ai-models-with-training-window).


# Auto Annotation

Magic annotation helps you generate bounding boxes automatically using AI models, so you can label datasets much faster.

It’s designed for a practical workflow:

1. Configure the model and prompts once
2. Auto-label one image to verify quality
3. Batch auto-label the whole dataset
4. Review and correct anything that’s wrong

***

## Requirements

{% hint style="info" %}
Magic annotation is fastest with an NVIDIA GPU.

* You will need a computer with an [NVIDIA GPU](/introduction/system-requirements)
* If an NVIDIA GPU is available, download torch GPU from Module Downloader to make the auto annotation run faster.
* Download the required AI modules from the Module Downloader window
  {% endhint %}

***

## First Look

![Magic annotation entry point in the Classes panel](/files/07MDsL0xeqH6fho6XA6t)

(Open Magic annotation with the ✨ button (or press T).)

You can open Magic annotation in two ways:

* Press the `T` key to auto-annotate the current image (uses your saved settings)
* Click the ✨ button in the **Classes** panel to open the **Magic annotation** settings dialog

{% hint style="info" %}
The first time you use Magic annotation, AugeLab Studio will ask you to configure your settings. These settings are remembered, and you can change them anytime by clicking the ✨ button.
{% endhint %}

***

## Before You Start (Recommended)

Magic annotation needs a dataset and a class list.

1. Load your dataset folder in the Image Annotation Window
2. Load (or create) your `classes.names` file

If you haven’t used the Annotation Window before, follow the main labeling guide first.

***

## Open the Magic annotation Dialog

1. Open **AI Tools** → **Image Annotation**
2. Load your dataset and class file
3. In the **Classes** panel, click the ✨ button

This opens the **Magic annotation Settings** dialog.

![Magic annotation settings dialog](/files/Jb3vV0OKJLgUzun2ZP6S)

***

## Step 1 — Choose a Model

In **Model Selection**, choose one of the supported detectors.

### Text-prompt models (recommended for custom classes)

These models detect objects using your class descriptions (text prompts):

* **Grounding DINO Tiny**: good default, faster
* **Grounding DINO Base**: more accurate, heavier (GPU strongly recommended)
* **OWLv2 Base Ensemble**: good general model
* **OWLv2 Large Ensemble**: more accurate, heavier (GPU strongly recommended)

Use these when your classes are not standard COCO classes, or when you want to describe the object in natural language.

### YOLO models (fast, but class matching matters)

YOLO models appear when the YOLO/OpenCV DNN feature is available:

* **YOLOv4 (COCO)**
* **YOLOv4 Tiny (COCO)**

These do **not** use text descriptions. They use COCO class names.

{% hint style="warning" %}
For YOLO (COCO) models, the COCO class names must match your dataset class names **exactly**. Example: if your class is `person`, it should be exactly `person` (not `human`).
{% endhint %}

### Custom YOLOv4 model

If you have your own YOLOv4 model, select **Custom YOLOv4 Model** and provide:

* Weights file (`.weights`)
* Config file (`.cfg`)
* Names file (`.names`)

***

## Step 2 — Set Thresholds

### Confidence Threshold

Controls how confident a detection must be to become a label.

* Higher values → fewer boxes, but usually cleaner
* Lower values → more boxes, but more false positives

A good starting point is **30%**.

### Grounding DINO: Box Threshold and Text Threshold

These appear only for Grounding DINO models:

* **Box Threshold**: how strict the box confidence should be
* **Text Threshold**: how strict the text-to-object matching should be

Guidance:

* If you get too many wrong boxes → increase **Text Threshold** first
* If boxes are sloppy or too wide → increase **Box Threshold**
* If you get no detections → lower thresholds gradually

***

## Step 3 — Write Better Class Descriptions (Text-Prompt Models)

If you selected a text-prompt model, you will see a **Class Descriptions** table.

![Class descriptions table](/files/ZIxYbw9hCGJ2A20EvJMn)

(Class Descriptions table (used as prompts).)

Why this matters: the description is the prompt the model uses to find your objects.

Good descriptions are:

* Visual and specific (color, shape, material)
* Grounded in your real images (background, lighting, orientation)

Examples:

* Instead of `bolt` → `silver bolt on a black conveyor belt`
* Instead of `cup` → `white paper cup, top view`
* Instead of `label` → `rectangular sticker label on a cardboard box`

You can also use:

* **Use Class Names** to reset prompts to the class names
* **Clear All** to start from scratch

{% hint style="info" %}
Tip: If two classes look similar, make the description emphasize what differentiates them. Example: `scratch on metal surface` vs `oil stain on metal surface`.
{% endhint %}

***

## Step 4 — Choose Annotation Mode (Important)

Magic annotation supports three modes for handling images that already have annotations:

* **Override**: replaces existing annotation files
* **Add**: appends new detections to existing annotations
* **Skip**: does not process images that already have annotations

Recommended usage:

* Choose **Override** when you’re starting fresh or re-labeling everything
* Choose **Add** when you want to *supplement* your existing labels
* Choose **Skip** when you’re polishing a partially labeled dataset and don’t want to risk overwriting work

***

## Run Magic annotation

### Annotate Current (one image)

Use **Annotate Current** first.

This is the safest way to validate that:

* your prompts are good
* thresholds are reasonable
* boxes look correct

If the results are not good, adjust prompts/thresholds and try again.

### Batch Annotate All (whole dataset)

When the current image looks good, click **Batch Annotate All**.

A progress dialog shows:

* current status (model loading / processing)
* progress bar
* estimated time remaining (ETA)

You can cancel at any time.

![Batch Magic annotation progress dialog](/files/gd6K5zK0nJI8Iy0H6nn0)

(Batch Magic annotation progress with ETA.)

***

## Review and Correct Results

Magic annotation is meant to accelerate labeling, not replace review.

After auto-labeling:

1. Quickly skim through images to catch obvious failures
2. Fix incorrect boxes (wrong class, wrong size)
3. Remove false positives
4. Add missed objects manually where needed

If you see repeated mistakes, stop and adjust prompts/thresholds, then rerun.

***

If your AI models aren't behaving as expected, use these quick-fix toggles to tune your performance.

<details>

<summary>🚫 "It annotates nothing" (Zero Detections)</summary>

When the AI is being too "shy" to label anything, it's usually a threshold or description issue.

* **Lower Confidence:** Drop the **Confidence Threshold** slightly (e.g., ).
* **Text Sensitivity:** For Grounding DINO, lower the **Text Threshold** to be less strict about word matching.
* **Be Specific:** Instead of "part," try "silver metal bolt" or "red plastic cap." Descriptions should be visual.
* **Check Lists:** Verify that your class list is actually loaded in the node settings and isn't empty.

</details>

<details>

<summary>📦 "Too many wrong boxes" (Ghost Detections)</summary>

If your screen is cluttered with false positives, you need to tighten the "strictness" of the model.

* **Raise Confidence:** Increase the **Confidence Threshold** to filter out low-certainty guesses.
* **Text Strictness:** Increase the **Text Threshold** to force a closer match between the image and your prompt.
* **Remove Ambiguity:** Avoid broad prompts like "object" or "item." If the AI is labeling shadows as "parts," specifically describe the part's unique colors or textures.

</details>

<details>

<summary>❓ "YOLO model doesn't detect my class"</summary>

Standard YOLO models are pre-trained on specific datasets.

* **COCO Standard:** Basic YOLO models only recognize the 80 COCO categories. Your labels must match exactly (e.g., `person`, `cell phone`, `chair`, `bottle`).
* **Custom Needs:** If you need to detect something specific (like a "scratched circuit board"), switch to a **Text-Prompt** model (like Grounding DINO) or train a **Custom YOLO** model.

</details>

<details>

<summary>🐌 "Processing is slow or laggy"</summary>

Vision models are computationally expensive.

* **First-Run Delay:** It is normal for the first run to be slow while models download and initialize in memory.
* **Model Size:** Grounding DINO Base and OWLv2 Large are high-accuracy but "heavy." Try a "Tiny" or "Small" variant for faster speeds.
* **Hardware:** Ensure AugeLab is utilizing your **GPU**. Running large AI models on a CPU will result in significant latency.

</details>

***

### 💡 Still stuck?

Try the **AI Assistant** in AugeLab Studio. Describe your specific camera view and what the boxes currently look like; it can often suggest the exact decimal value for your thresholds.

Would you like me to create a **"Threshold Cheat Sheet"** table that explains exactly what Confidence vs. Text thresholds do?

***

## Notes

* Magic annotation settings are saved and reused when you press `T`.
* If your plan/licensing has limits for offline Magic annotation, the tool will prevent batch processing after the limit is reached.


# Augment Dataset

Dataset augmentation creates new training images by applying controlled transformations to your existing labeled images while keeping bounding boxes perfectly aligned.

Used correctly, augmentation helps your model **generalize** (perform well on new images). Used incorrectly, it can **make training worse** by teaching the model unrealistic patterns.

***

## ⚖️ What Augmentation Is (and Is Not)

![Augmentation Examples](/files/6INxMtVaWbShM7neZMqd)

| **Augmentation IS...**                          | **Augmentation IS NOT...**                           |
| ----------------------------------------------- | ---------------------------------------------------- |
| A way to simulate lighting, rotation, or noise. | A substitute for missing camera angles or products.  |
| A tool to improve robustness in small datasets. | A fix for incorrect or "sloppy" initial labels.      |
| A method to reduce overfitting.                 | A guarantee of better results (over-doing it hurts). |

***

## Strategic Usage

<details>

<summary>When Augmentation Is Helpful</summary>

* **Small Datasets:** You have few images per class.
* **Environmental Variation:** You expect shifts in lighting (day/night), glare, or motion blur.
* **Overfitting:** Training accuracy is high, but real-world performance is low.
* **Rare Classes:** Some objects appear infrequently in your real data.

</details>

<details>

<summary>When Augmentation Is Unnecessary (or Risky)</summary>

* **Diverse Real Data:** You already have thousands of varied, real-world images.
* **Subtle Features:** Your inspection depends on tiny scratches or textures that blur/noise might destroy.
* **Stable Environments:** The lighting, camera, and product positions never change. For example, a fixed position object does not need rotation augmentation.

</details>

{% hint style="warning" %}
**Disk Space Alert:** Augmentation generates new physical files. Enabling many options can cause your dataset folder size to explode. **Run augmentation only after finishing manual labels and keeping a backup.**
{% endhint %}

***

## 🛠️ How the Augmentation Window Works

In the Image Annotation Window, navigate to **Tools** → **Augment Dataset**.

![Augmentation Selector window](/files/pINqJ0xlE5Fyw8x3uPbK)

### Interface Breakdown

| Section              | Function                                                                                              |
| -------------------- | ----------------------------------------------------------------------------------------------------- |
| **Quick Presets**    | One-click configurations to apply sensible starting defaults. ![Presets](/files/WFRXhPTpEesEfu9BGA7w) |
| **Quick Toggles**    | Enable/disable entire groups (e.g., Color, Noise) quickly. ![Toggles](/files/x3E4yfXXmqnY60AvGiKL)    |
| **Detailed Options** | Fine-tune intensity for Brightness, Contrast, Blur, Noise, and Perspective tweaks.                    |

***

## A Safe, Practical Workflow

1. **Clean Baseline:** Finish manual labeling first (or at least a clean subset).
2. **Backup:** Duplicate your dataset folder.
3. **Start Small:** Use a preset or minimal manual settings.
4. **Visual Audit:** Open the generated folder and check:
   * Are the bounding boxes still centered on the objects?
   * Do augmented images still look realistic?
5. **Train & Compare:** Compare the results of a model trained with and without augmentation.

***

## ❓ Troubleshooting

<details>

<summary>📉 "My model got worse after augmentation"</summary>

This often means the augmentation didn't match reality.

* **Try:** Reduce augmentation intensity.
* **Try:** Disable transforms that create unrealistic images (e.g., don't use 180° rotation if parts are always upright).

</details>

<details>

<summary>🖼️ "Boxes look wrong on augmented images"</summary>

* **Try:** Reduce geometric transforms (rotation/perspective).
* **Try:** Validate that your labels were tight and correct before augmentation.

</details>

<details>

<summary>📂 "It generated too many files"</summary>

* **Try:** Disable most transforms and keep only the ones you really need.
* **Try:** Use augmentation on a smaller subset of images.

</details>


# After Annotation

You've finished annotating your dataset! 🎉

<figure><img src="/files/ktbnETPTd8ad4lPImBgX" alt="Annotation Finished" width="400"><figcaption><p>YAY!</p></figcaption></figure>

A high-quality dataset is consistent. You may easily follow this document to perform a quick audit of your annotations before moving to training.

***

## Quick-Check via Dataset Filters

In the Image Annotation Window, use the filter dropdown to isolate specific labeling states.

| Filter        | Logic               | What to look for                                   |
| ------------- | ------------------- | -------------------------------------------------- |
| **All**       | Total dataset       | General overview of project volume.                |
| **Annotated** | ≥1 Bounding Box     | Ensure boxes are tight and classes are correct.    |
| **Empty**     | Background/Negative | **Critical:** Confirm these truly have no objects. |
| **Excluded**  | No annotation file  | Ensure no usable data was accidentally hidden.     |

{% hint style="info" %}
**Common Pitfall:** Having "Empty" images that actually contain objects will confuse the model. If an object is there, it must be labeled or the image must be "Excluded."
{% endhint %}

***

## Quick Review: Keyboard Shortcuts

These shortcuts allow for rapid auditing without leaving the canvas.

### Navigation & Class Selection

* `D` / `A`: Next / Previous image.
* `Shift + D` / `Shift + A`: Jump forward/back by 10 images.
* `S` / `W`: Next / Previous class.
* `Shift + S` / `Shift + W`: Jump classes by 3.
* `H` (Hold): Temporarily hide annotations to see the raw image.

### Labeling & File Management

* `O`: **Mark as Background** (Creates/clears an empty annotation file).
* `P`: **Exclude Image** (Removes the annotation file).
* `X`: Remove last bounding box.
* `Shift + C`: Clear all boxes on the current image.
* `M`: Move image + annotation to a `/moved` subfolder (Folder Mode).
* `Shift + Delete`: **Permanently delete** image + annotation.

***

## Advanced Analysis Tools

### A) Class Frequency Analysis

Open **Tools → Class Frequency Analysis** to visualize your data distribution.

* **Rare Classes:** If a class is significantly lower than others, the model may ignore it.
* **Dominant Classes:** If one class makes up the bulk of the data, the model may over-predict it.

If you find imbalances, consider collecting more data for rare classes or removing some examples of dominant classes with redundant images.

Another option is to use data augmentation techniques to artificially increase the variety of underrepresented classes.

### B) Pattern Recognition

Watch for these "Annotation Quality" issues during your review:

* **Loose Boxes:** Too much background noise inside the box.
* **Inconsistent Style:** Mixing tight and loose boxes across the same class.
* **Missing Negatives:** Not enough "Empty" images to teach the model what *isn't* an object.

***

## Audit Routine

1. **Filter to Annotated:** Review \~20–50 images across the entire set (not just the first page).
2. **Filter to Empty:** Review \~10–20 images to ensure they are truly empty.
3. **Spot-check Excluded:** Ensure no high-quality data is sitting idle.
4. **Edge-Case Pass:** Search for the smallest objects, worst glare, and heaviest motion blur.

***

## Validation Set

Pick 30–100 images or video clips that represent "Real World" challenges (bad lighting, clutter, etc.).

Keep these labeled perfectly. Use this set as your final "reality check" before deploying any model to production.

{% hint style="danger" %}
**Backup Reminder:** Always duplicate your dataset folder before running batch operations or mass-deletions.
{% endhint %}


# Train Object Detection(YOLO) Models

The **Object Detection Training Window** trains YOLO-based object detection models using your annotated dataset.

{% hint style="info" %}
Training is best with a compatible [Nvidia GPU](/introduction/system-requirements) and the AugeLab Studio AI/GPU package enabled.

If the training window is disabled or shows missing runtime/dependency errors, open the [Module Downloader Window](/augelab-studio-interface/external-features/module-downloader) and install the required AI tools.
{% endhint %}

<figure><img src="/files/CKJe1oxIGAP2mTLLxzab" alt=""><figcaption></figcaption></figure>

## Getting Started

1. Launch AugeLab Studio.
2. Open `AI Tools` → [**Object Detection Training Window**](/key-features/train-custom-ai-models-with-training-window)
3. Prepare:
   * a **dataset folder** and **class names file** prepared with AugeLab Studio’s [Annotation Tool](/key-features/annotate-data-for-object-detection/annotation-window-how-to)
   * or a **dataset folder** containing images and YOLO label files (`.txt`) in the **same folder**
   * a **class names file** in `.names` format (one class name per line)

{% hint style="info" %}
The training window scans your dataset and shows **Dataset Analytics** (total images, annotated/unannotated counts, and the class list) to help you catch mistakes early.
{% endhint %}

## 1) Configuration

The current training window uses a simple **Configuration** panel instead of a menu-only workflow.

### Select Dataset Folder

Choose the folder that contains your training images.

<figure><img src="/files/0vGFxQhkn2Mn0GfjT3Iy" alt=""><figcaption></figcaption></figure>

Notes:

* Only files in this folder are used (keep your training images in one folder)
* Supported image extensions include: `.jpg`, `.jpeg`, `.png`, `.bmp`, `.tiff`, `.tif`

### Select Class Names File (`.names`)

Choose your class list file.

<figure><img src="/files/3aDmlsICBUL7tA5PyqcD" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
If the `.names` file is empty, the training window will treat it as an error. Make sure it contains one class name per line.
{% endhint %}

### Model Type

Choose a model variant from **Model Type**.

<figure><img src="/files/q0HW3QuFY2zxe8uRz5Cc" alt=""><figcaption></figcaption></figure>

In general:

* **Robust Ones** variants are slower but can reach higher accuracy, YOLOv4-Scaled is a good default
* **Fast** variants train faster and are easier on low-spec PCs
* **Micro / Nano** are designed for very small / edge-device style models

### Optional: Custom Weights

You can start from custom/pretrained weights (Darknet `.weights` / backbone `.conv.*`).

<figure><img src="/files/6D7MAwQOj9w5fbqMJdsW" alt=""><figcaption></figcaption></figure>

Good use cases:

* Continuing a previous run
* Faster convergence on similar datasets

{% hint style="warning" %}
If you change **Model Type**, it’s safest to clear and re-select weights that match the chosen model.
{% endhint %}

## 2) Advanced Settings

Advanced Settings allows you to tune training behavior (memory use, speed, and accuracy).

<figure><img src="/files/gGIJ68VCuNSX8tgmC1vb" alt=""><figcaption></figcaption></figure>

The most important settings:

* **Dataset Split Ratio (Train/Val)**: how much data is used for validation (affects mAP reporting)
* **Network Input Size (Width/Height)**: bigger can help small objects, but uses more VRAM and slows training
* **Batch Size / Subdivisions**: main knobs for GPU memory errors
  * If you see “Out of Memory”, **increase subdivisions** or **decrease batch size**
* **Recalculate Anchors**: can improve results on custom datasets (recommended for new datasets)
* **Calculate Optimal Network Size**: optional auto-selection helper
* **GPUs to Use**: for multi-GPU systems (e.g., `0` or `0,1`)
* **mAP During Training**: shows accuracy progress but can slow training a bit
* **Clear Previous Training**: start fresh vs. resume
* **Live Augmentation Options**: applies on-the-fly variations during training (does not create extra files)

<figure><img src="/files/bVdBrw0Pd6xK0HUXJ3iF" alt=""><figcaption></figcaption></figure>

## 3) Start / Stop Training

Once Dataset + Classes are valid, the main button becomes active.

1. Click **Start Training**
2. Monitor:
   * the **Log** area (console output)
   * the **Training Chart** window (loss / mAP)
3. Click **Stop Training** to terminate the process

<figure><img src="/files/sB634WyB0lN5xDaMMBvU" alt="" width="563"><figcaption><p>Training Logging</p></figcaption></figure>

{% hint style="warning" %}
Closing the training window while training is running will terminate the training process.
{% endhint %}

## After Training

When training finishes (or you stop it), check the output directory referenced in the log/status messages.

Next steps:

* Load your trained model into your workflow (inference)
* Validate results on a holdout set or real camera footage


# When to Stop Training

Training doesn’t need to run “forever”. In real projects, the best results come from stopping at the right moment:

* not too early (model hasn’t learned yet)
* not too late (model starts to overfit / memorize)

{% hint style="info" %}
In AugeLab Studio, training usually ends when it reaches the configured **max iterations**, or when you click **Stop Training**. The Training Chart helps you decide whether it’s worth continuing.
{% endhint %}

If this is your first training, start with the [Starter Checklist](#starter-checklist).

## Monitor Training Progress <a href="#monitor-training-progress" id="monitor-training-progress"></a>

During training, monitor the progress of the model and watch the relationship between:

* Loss
* mAP
* IOU
* Iterations

Loss and mAP are shown on a chart like below:

<figure><img src="/files/PDnVAECN6LdxEW4ypgm4" alt="Good training example chart"><figcaption><p>Example: Good training (loss decreases, mAP increases then plateaus)</p></figcaption></figure>

{% hint style="warning" %}
All metrics can wildly vary by:

* Data variety
* Data size
* Annotation accuracy
* Model size

Numbers below are only provided for setting an initial ground for newcomers.
{% endhint %}

### Quick Rule (what usually works) <a href="#quick-rule" id="quick-rule"></a>

If you only remember one rule:

Stop when validation mAP stops improving for a long time, or when it starts going down while loss keeps going down.

That second case is the classic sign of overfitting.

### Common Training Patterns (cheat sheet) <a href="#common-patterns" id="common-patterns"></a>

These patterns are common in real use. For each one, look at the chart first, then read the explanation.

{% hint style="info" %}
These example charts are generated for training/documentation purposes. In your repository, place them under the .assets/ folder next to this page.
{% endhint %}

#### Insufficient data <a href="#examples-insufficient-data" id="examples-insufficient-data"></a>

<figure><img src="/files/WnAKIwqKfHWzWMOoFTa9" alt="Insufficient data example chart"><figcaption><p>Insufficient data: too few points / too short run (noisy early metrics)</p></figcaption></figure>

Explanation:

* What it means: you don’t have enough signal yet to trust the trend.
* Likely causes: too few images, too short run, weak/too small validation split.
* What to do: train longer; add data; ensure validation exists and includes real variety.

#### Low variance <a href="#examples-low-variance" id="examples-low-variance"></a>

<figure><img src="/files/tzexJN7KjF4AaDBlcdAR" alt="Low variance example chart"><figcaption><p>Low variance: loss plateaus and mAP barely improves</p></figcaption></figure>

Explanation:

* What it means: the model learns the “easy repetition” quickly, then stops getting new information.
* Likely causes: repetitive dataset (same background/angle/light), missing negatives, missing edge cases.
* What to do: add variety (angles, backgrounds, lighting), add negatives, capture hard cases on purpose.

#### Overtraining <a href="#examples-overtraining" id="examples-overtraining"></a>

<figure><img src="/files/LduyNB0HqvauUuCh58M9" alt="Overtraining example chart"><figcaption><p>Overtraining: loss keeps improving, but mAP peaks (even very high) and then degrades</p></figcaption></figure>

Overtraining is not always catastrophic, but it usually indicates memorization rather than generalization. For strict environments (fixed camera, fixed lighting), it is acceptable.

* What it means: the model is getting better at the training set, but worse at validation (memorization).
* Likely causes: not enough variety, too-small validation, duplicates/near-duplicates.
* What to do: stop and keep best weights; add more variety; increase validation split; remove duplicates.

#### Model not learning <a href="#examples-not-learning" id="examples-not-learning"></a>

<figure><img src="/files/ciTTvQDwPDqU7Lenyp1W" alt="Model not learning example chart"><figcaption><p>Model not learning: loss stays high/flat, mAP stays near zero</p></figcaption></figure>

Explanation:

* What it means: training is not progressing in a meaningful way.
* Likely causes: wrong labels/classes, class IDs mismatch, broken annotation format, incorrect config/settings.
* What to do: verify `.names` order vs label IDs; spot-check labels; confirm YOLO format; adjust training settings.

#### Corrupted dataset <a href="#examples-corrupted" id="examples-corrupted"></a>

<figure><img src="/files/npxJRiNPstdw7xiy9s3d" alt="Corrupted dataset example chart"><figcaption><p>Corrupted dataset: unstable loss spikes and erratic mAP</p></figcaption></figure>

Explanation:

* What it means: training is being disrupted by inconsistent or broken data.
* Likely causes: corrupted image files, invalid labels, mixed sources/resolutions, “empty-but-contains-objects” images.
* What to do: run dataset checks; remove corrupted data; fix label format; re-export a clean set.

#### Good training <a href="#examples-good" id="examples-good"></a>

<figure><img src="/files/PDnVAECN6LdxEW4ypgm4" alt="Good training example chart"><figcaption><p>Good training: steady learning and a stable high plateau</p></figcaption></figure>

Explanation:

* What it means: healthy learning and generalization.
* Likely causes: consistent labels + enough variety.
* What to do: stop when mAP plateaus; validate on real footage / a “golden set”; deploy best weights.

### Loss <a href="#loss" id="loss"></a>

Loss is a training-fit signal. It represents how well the model is fitting the training batches.

Loss is useful, but it can be misleading:

* Loss can keep decreasing even when the model is already overfitting.
* Loss does not guarantee “real-world performance”.

{% hint style="info" %}
Loss alone is not enough to judge accuracy. Use [mAP](#map) to understand generalization on validation data.
{% endhint %}

#### \*\*2.0 ≥\*\* Loss <a href="#id-20-loss" id="id-20-loss"></a>

Often indicates “learning has started”, but quality may still be poor. Use it as a sign that the pipeline works, not as a finish line.

{% hint style="warning" %}
As shown in the graph above, loss values around 2.0 may not produce accurate models.
{% endhint %}

#### \*\*1.0 ≥\*\* Loss <a href="#id-10-loss" id="id-10-loss"></a>

Commonly a usable baseline on many focused datasets.

#### \*\*0.5 ≥\*\* Loss <a href="#id-05-loss" id="id-05-loss"></a>

Often indicates a well-fit model on a clean, consistent dataset. After this point improvements can be slow, and overfitting risk increases.

<details>

<summary>Loss thresholds are not universal (why)</summary>

Loss values depend on model architecture, image size, classes, label noise, augmentation, and dataset complexity. Use loss thresholds to build intuition, not as a universal pass/fail.

</details>

### mAP <a href="#map" id="map"></a>

The mAP (mean average precision) metric combines both precision and recall to provide a comprehensive evaluation of the model's accuracy in detecting objects in an image.

It is calculated by evaluating predictions against ground-truth labels at specific IoU thresholds (exact details depend on the training backend/settings).

{% hint style="warning" %}
mAP is only as good as your validation set. If validation images are too few, too “clean”, too similar to training, or mislabeled, mAP can look great while the model fails in production.
{% endhint %}

Practical interpretation:

* A stable plateau is often more important than chasing the last +1%.
* Very high mAP (like 95–99%) on a small or repetitive dataset is a common overfitting trap.
* If mAP peaks then drops, see [Over-Fitting](#over-fitting).

### IOU <a href="#iou" id="iou"></a>

IOU (Intersection over Union) measures the overlap between predicted and true bounding boxes for individual object detections. mAP evaluates the overall performance of the object detection model across all object categories, considering both precision and recall.

{% hint style="info" %}
Higher the IOU value, tighter the predicted box is.
{% endhint %}

You can track each IOU in Training Window loggings:

<figure><img src="/files/MqsTImBfLKSgVicfD1kB" alt=""><figcaption></figcaption></figure>

## Fine Tuning <a href="#fine-tuning" id="fine-tuning"></a>

### Training Time <a href="#training-time" id="training-time"></a>

Define a maximum training time budget based on available computational resources and project constraints. If the model does not achieve satisfactory performance within the allocated time, consider stopping training and exploring other approaches such as:

* Manually analyze annotation accuracy
* Check class variety
* Choose different model sizes and batch sizes
* Increase database size

### Over-Fitting <a href="#over-fitting" id="over-fitting"></a>

Avoid overfitting by monitoring how mAP behaves over time.

The most reliable “real life” overfit signal is:

loss decreases, but mAP peaks and then gets worse.

Overfitting is not always “catastrophic” on very constrained, fixed-camera setups. But if you care about robustness (different lighting, different shifts, different backgrounds), overfitting will show up quickly.

What usually helps:

* Add more variety (new days, new lighting, new backgrounds)
* Add negatives that look like your real environment
* Tighten label consistency (same style across labelers)
* Increase validation split so mAP is harder to “cheat”

<figure><img src="/files/NumGWCcgXPuUCLjdoeHq" alt="" width="563"><figcaption></figcaption></figure>

<figure><img src="/files/LduyNB0HqvauUuCh58M9" alt="Overtraining example chart"><figcaption><p>Overtraining example: mAP peaks then drops while loss continues decreasing</p></figcaption></figure>

### Balancing Time and Performance <a href="#balancing-time-and-performance" id="balancing-time-and-performance"></a>

Balance the training time with the desired model performance. In some cases, additional training iterations may improve performance, but the returns may diminish over time. Weigh the benefits against the computational cost and the urgency of the project.

Usually, depending on class numbers and database size, training process length can vary between a day or a week.

## Starter Checklist <a href="#starter-checklist" id="starter-checklist"></a>

Database:

* [ ] Labels are consistent (box style + class meaning)
* [ ] Dataset has real-world variety (lighting, angles, backgrounds)
* [ ] You have enough examples per class to learn (more is better; start small, then improve)
* [ ] (Optional) Augmentation is enabled *after* labels are correct

Model:

* [ ] Chosen a model size that meets FPS requirements
* [ ] Right model for the right [system requirements](/introduction/system-requirements) and CUDA compatibility, GPU memory.
* [ ] Batch size according to GPU memory (use subdivisions to avoid OOM)

Training (stop if):

* [ ] [mAP](#map) plateaus for a long time (diminishing returns)
* [ ] [mAP](#map) falls while [Loss](#loss) continues falling (overfitting)
* [ ] You hit your time budget and results are “good enough” to test on real footage

<details>

<summary>Fast debugging checklist (when things look wrong)</summary>

1. Spot-check 20–50 images across the dataset (not just the first page)
2. Confirm class mapping:

* `.names` file order matches label IDs
* no missing/extra classes

3. Spot-check label files:

* YOLO format: `class x_center y_center width height` (normalized)
* boxes are in-bounds and not zero-sized

4. If mAP looks “too good to be true”:

* validation split may be too small or too similar to training
* you may have duplicates / near-duplicates

5. If training is unstable or OOM:

* increase subdivisions or reduce batch
* temporarily reduce input resolution to debug

</details>


# After Training

You are now close to deploying your model — it’s the point where you turn “numbers” into a model that actually works in your real workflow.

***

## 1) Find Your Training Output (weights + config + names) <a href="#id-1-find-output" id="id-1-find-output"></a>

When training ends, the log/status messages show where outputs are written.

In AugeLab Studio, training outputs are typically created in a folder named:

XXX\_config

right next to your dataset folder (XXX is your dataset folder name).

Typical structure:

```
XXX_config/
    XXX.names
	XXX.cfg
	backup/
		XXX_last.weights  (if available)
		XXX_best.weights  (if available)
```

At minimum, you should keep these together:

* Weights file: `.weights` (sometimes there is also a best vs last style file)
* Config file: `.cfg`
* Class names file: `.names`

{% hint style="warning" %}
Do not rename/reorder classes in your `.names` file after training unless you also remap label IDs. Class order must match label IDs.
{% endhint %}

<details>

<summary>Which weights should I use: best vs last?</summary>

If your training reports mAP during training, many YOLO/Darknet workflows keep a “best so far” checkpoint.

* Use best when mAP improved and then later dropped (overfitting).
* Use last if training ended while mAP was still improving and stable.

If you don’t have a “best” file, pick the final weights first, then validate.

</details>

***

## 2) Validate the Model Before You Deploy <a href="#id-2-validate-before-deploy" id="id-2-validate-before-deploy"></a>

Before you wire the model into production logic, do a fast validation pass.

Recommended validation sets:

* Validation set: 30–100 images that represent real life (good + bad lighting, blur, clutter, edge cases)
* Short videos or images: short video from the real camera (if you will deploy on a fixed camera)

What you are looking for:

* The model detects the right object consistently
* The boxes are “good enough” for your logic (not necessarily perfect)
* False positives are acceptable (or can be filtered)
* Rare-but-important cases are detected

{% hint style="warning" %}
High mAP can still fail in production if your validation split was too small or too clean. The validation set / real footage check is what prevents that.
{% endhint %}

***

## 3) Load the Model Into a Studio (Inference) <a href="#id-3-load-into-graph" id="id-3-load-into-graph"></a>

In AugeLab Studio, the usual next step is to build (or update) a `.pmod` scenario that runs inference.

### A) Use “Object Detection - Custom” (recommended) <a href="#id-3a-use-object-detection-custom" id="id-3a-use-object-detection-custom"></a>

Use this node when you want to run your own YOLO/Darknet-trained model inside a workflow.

Workflow:

1. Add Object Detection - Custom to your graph (AI Applications category).
2. In the block UI:
   * Click Open Weight File and select your `.weights`
   * Click Open Config File and select your `.cfg`
   * Click Open Class File and select your `.names`
3. Select which classes you want to detect (checkbox list).
4. Set Confidence Threshold (start around 0.5–0.8 and tune).
5. Connect an image source to the block input and preview the output image.

Outputs you can use in your logic:

* Output image with drawn detections
* Object Count
* Object Locations / Sizes
* Object Classes
* Rectangles

{% hint style="info" %}
If “Object Detection - Custom” is not available, your build may not have CUDA/OpenCV DNN support enabled. Try the CPU block below, or install the required modules from the Module Downloader (see [ai-training.md](/key-features/train-custom-ai-models-with-training-window)).
{% endhint %}

### B) Use “Object Detection - Custom (CPU)” (fallback) <a href="#id-3b-use-object-detection-custom-cpu" id="id-3b-use-object-detection-custom-cpu"></a>

Use this block when you want the same workflow but without GPU acceleration.

* It uses CPU inference, so it will be slower.
* The setup is the same: weights + cfg + names.

***

## 4) Tune Thresholds (what actually matters) <a href="#id-4-tune-thresholds" id="id-4-tune-thresholds"></a>

Most “deployment quality” improvements come from threshold tuning, not from running training longer.

Start with these practical steps:

* Increase confidence threshold if you see too many false positives.
* Decrease confidence threshold if you miss objects.
* Evaluate on the golden set and at least one real camera clip.

{% hint style="warning" %}
Do not tune on a single image. Always tune on a small set. Otherwise you will “overfit your threshold” to one scene.
{% endhint %}

***

## 5) Package for Sharing / Reproducibility <a href="#id-5-package" id="id-5-package"></a>

If you want the model to be usable later (or by someone else), package it intentionally.

Recommended folder layout:

```
my_model_release/
	model.weights
	model.cfg
	classes.names
	README.txt
	validation_set/   (optional)
```

What to write in the README:

* What dataset the model was trained on (version/date)
* What classes mean (if ambiguous)
* Recommended confidence threshold range
* Known failure cases (glare, tiny objects, extreme occlusion)

{% hint style="info" %}
If your `.pmod` scenario references these resources, consider keeping them as relative project resources so the scenario remains portable. See also: [headless-studio](/key-features/headless) (missing-resource load behavior).
{% endhint %}

***

## 6) If It Fails in Production (what to do next) <a href="#id-6-if-it-fails" id="id-6-if-it-fails"></a>

When a model fails after deployment, the fix is usually one of these (in this order):

1. Collect the failures (save frames that show the miss/false-positive)
2. Label them correctly
3. Retrain or fine-tune with the new data

This is how models get robust.

<details>

<summary>Common failure modes and the fastest fix</summary>

* False positives on background texture → add negatives from that exact environment
* Misses on small objects → increase input size (if GPU allows) and collect more small-object examples
* Misses under glare/blur → add those cases intentionally to the dataset (do not rely only on augmentation)
* Boxes are consistently too loose/tight → fix annotation style consistency, then retrain

</details>


# Create Plugins

Develop custom nodes using the power of Python.

## First Look <a href="#first-look" id="first-look"></a>

<figure><img src="/files/JAY4L21O0itelzwagyMp" alt=""><figcaption></figcaption></figure>

Designer Window is the fastest way to go from “I need a special node” → “I can use it in my scene”.

You write (or generate) a small Python class (a `Block`), press **CREATE BLOCK**, and it shows up in the Custom Blocks list.

{% hint style="info" %}
If you like starting from a working template, open [Coding Reference](/key-features/create-plugins-with-designer-window/coding-reference) and copy the sample block.
{% endhint %}

### Quick Start <a href="#generate-block-script-button" id="generate-block-script-button"></a>

Follow these steps once, then come back and explore the details.

<figure><img src="/files/YrLhCvMj4OMuyrcDNguV" alt=""><figcaption></figcaption></figure>

1. Open **Designer Window**.
2. Keep the default script, or paste your own or prompt your way with AI agent.
3. Make sure the script ends with `add_block(...)`.
4. Press **CREATE BLOCK**.
5. Find your block in the **Custom Blocks** list and drag-drop it into a scene.

{% hint style="success" %}
Tip: Keep your socket names stable. It makes updating a block much smoother.
{% endhint %}

### Code Editor <a href="#code-editor" id="code-editor"></a>

The big editor area is the source of truth for your block.

Your script must include (minimum):

* `from studio.custom_block import *`
* A class that inherits from `Block`
* A matching `op_code`
* A final `add_block(MyBlock.op_code, MyBlock)` line

{% hint style="warning" %}
On creation, AugeLab Studio normalizes indentation by replacing tabs with 4 spaces before saving.
{% endhint %}

### AI Assistant <a href="#parameter-settings-section" id="parameter-settings-section"></a>

<figure><img src="/files/8RwZM5RIF2HM618uVwm3" alt=""><figcaption></figcaption></figure>

At the bottom of Designer Window you can:

* Write a prompt (example: “Write a block that converts a BGR image to grayscale”)
  * Be descriptive for the best results.
  * As of now, custom blocks assistant is free to use.
* Choose a model from the dropdown
* Press **Submit** to generate code into the editor

This is meant to get you started quickly. You’re always in control—review and edit the code before you press **CREATE BLOCK**.

{% hint style="info" %}
AI agent remembers your previous prompts, you can ask for improvements or changes.
{% endhint %}

{% hint style="warning" %}
The AI feature may be unavailable depending on license, connectivity, or server status.
{% endhint %}

### Updating an Existing Block <a href="#block-configuration" id="block-configuration"></a>

When you press **CREATE BLOCK** again:

* The file is overwritten.
* The Custom Blocks list entry is refreshed.
* Studio attempts a best-effort *safe replace* in open scenes (it tries to preserve connections).

{% hint style="info" %}
If you renamed sockets, Studio may reconnect by index as a fallback. That’s why stable socket names matter.
{% endhint %}

<details>

<summary><strong>Advanced: What CREATE BLOCK actually does</strong></summary>

* Studio finds the first class inheriting from `Block` and uses its class name as the block name.
* Your script is saved as `<BlockName>.py` under the Marketplace custom blocks folder.
* Studio imports `custom_blocks.<BlockName>` and instantiates it once to validate it.
* If everything looks good, the block becomes available in the Custom Blocks list.

{% hint style="info" %}
File location: `.../AugeLab Studio/marketplace/custom_blocks/<BlockName>.py`
{% endhint %}

{% hint style="warning" %}
If import/validation fails, Studio shows the error and removes the file. Fix the script and try again.
{% endhint %}

</details>

### Reloading Blocks into Designer Window <a href="#reloading-blocks-into-designer-window" id="reloading-blocks-into-designer-window"></a>

To edit an existing block, right-click its name in the Custom Blocks list and choose **Load into Designer Window**.

{% hint style="warning" %}
Loading a block into Designer Window only works for editable user scripts (`.py`). Compiled/encrypted blocks (for example `.pyd` or PyArmor-protected scripts) are intentionally blocked.
{% endhint %}

<figure><img src="/files/LLADaPjVNiik0frovNkB" alt=""><figcaption><p>Load Existing Block</p></figcaption></figure>

{% hint style="info" %}
“Reload” updates the Custom Blocks list (adds new files / removes deleted files). It does not hot-update nodes already placed in scenes.

<img src="/files/V2YmZtl9qakRWFxffOBB" alt="Refresh Block List" data-size="original">
{% endhint %}


# Components

Components are interactive widgets that let users configure parameters (inputs) or view results inside your custom block.

{% hint style="info" %}
Components are constructed by keyword arguments.
{% endhint %}

#### Generic Arguments <a href="#generic-arguments" id="generic-arguments"></a>

Generic arguments apply to all custom components. You can provide them as keyword arguments to constructors.

Instead of repeating them under every component, here is a small “pyi-style” reference:

```python
# Common keyword arguments (supported by all components)
class _CommonComponentKwargs:
    def __init__(
        self,
        *,
        tool_tip: str = "",
        enabled: bool = True,
        hidden: bool = False,
        fixed_width: int = 0,
        fixed_height: int = 0,
        minimum_width: int = 0,
        minimum_height: int = 0,
        maximum_width: int = 0,
        maximum_height: int = 0,
        serializable: bool = True,
        stylesheet: str = "",
        font_size: int = -1,
        font_bold: bool = False,
        alignment: str = "AlignLeft",
        **kwargs,
    ) -> None: ...
```

Notes:

* `tool_tip` controls the tooltip shown when hovering the component.
* `fixed_width` / `fixed_height` are pixels. Use `0` for “auto”.
* Some components override defaults (for example `DropDown` is non-serializable; `Image` is non-serializable by default).

### Text Input <a href="#text-inputx20" id="text-inputx20"></a>

Allows users to input text/number through a single line.

<figure><img src="/files/99A00bjsOtKy6VI2L56S" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class TextInput:
    def __init__(
        self,
        *,
        text: str = "",
        place_holder: str = "",
        check_type: type = str,
        password: bool = False,
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def value(self) -> str: ...

    @property
    def text(self) -> str: ...  # same as .value

    def toInt(self) -> int: ...
    def toFloat(self) -> float: ...
```

Notes:

* Use `check_type=int` or `check_type=float` to restrict what the user can type.
* Use `.toInt()` / `.toFloat()` when you want conversion errors to surface with a clear exception (no silent fallback).

<mark style="color:blue;">**Example:**</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param["text1"] = TextInput(
            text="5",
            place_holder="Enter an integer",
            check_type=int,
            tool_tip="Defines constant",
        )
    
    def run(self):
        ...
        constant: int = self.param["text1"].toInt()
        ... 
```

### Drop Down List <a href="#drop-down-list" id="drop-down-list"></a>

Drop down lists allows users to choose an option from a provided list of texts.

<figure><img src="/files/v597Attey0h6R1wgZjmh" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class DropDown:
    serializable: bool = False

    def __init__(
        self,
        *,
        items: list[str] | dict[str, object] = ["item1", "item2", "item3"],
        selected_index: int = 0,
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def selected_item(self) -> str: ...

    @property
    def selected_index(self) -> int: ...
```

Notes:

* If you pass `items` as a `dict[str, object]`, you can access the mapped value via `.getCurrentMatch()`.

{% hint style="warning" %}
`DropDown` is currently marked as non-serializable. If you want the selection to persist in saved scenarios, store `selected_index` yourself (for example with `register_ser_value()` or `serialize_node()`).
{% endhint %}

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param['drop_down'] = DropDown(items=['Method 1', 'Method 2', 'Method 3'], 
                                        tool_tip='Choose Method')
    
    def run(self):
        ...
        method_index: int = self.param['drop_down'].selected_index
        method_name: str = self.param['drop_down'].selected_item
        if method_index == 0:
            ... 
```

### Label <a href="#label" id="label"></a>

Labels are simple text based components to statically or dynamically show text on your custom block.

<figure><img src="/files/BifhOn3E2iYjYDQRoFhR" alt=""><figcaption></figcaption></figure>

They are also used to provide information about interactable components:

<figure><img src="/files/8F15oflGsGiaDtcR5s16" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class Label:
    def __init__(self, *, text: str = "", tool_tip: str = "", **kwargs) -> None: ...
    def set_text(self, text: str) -> None: ...
```

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param['label'] = Label(text='Result is: Not Set', 
                                        tool_tip='Shows mean value')
    
    def run(self):
        ...
        self.param['label'].set_text(f'Result is: {n}')
        ... 
```

### Slider <a href="#slider" id="slider"></a>

Restricts user input to range of numbers.

<figure><img src="/files/E6qZ6jexKs8Xc5fzORiB" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class Slider:
    def __init__(
        self,
        *,
        min: int = 0,
        max: int = 100,
        val: int = 50,
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def value(self) -> float | int: ...
```

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param['slider'] = Slider(min=-5, max=5, val=3)
    
    def run(self):
        ...
        threshold: int = self.param['slider'].value
        ... 
```

### Slider Labeled <a href="#slider-labeled" id="slider-labeled"></a>

Same as [Slider](#slider) but adds a label that automatically shows which value is shown in component.

<figure><img src="/files/OKIomB9c6AHIJy4plZnh" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class SliderLabeled:
    def __init__(
        self,
        *,
        min: int = 0,
        max: int = 100,
        val: int = 50,
        label: str = "Value",
        multiplier: float | int = 1,
        add: float | int = 0,
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def value(self) -> float | int: ...

    @property
    def modifiedValue(self) -> float | int: ...
```

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param['threshold_odd'] = SliderLabeled(min= -5, max= 5, val= 3, label="Value", multiplier = 2, add = -1)
    
    def run(self):
        ...
        threshold_odd: int = self.param['threshold_odd'].modifiedValue
        ... 
```

### CheckBox <a href="#checkboxx20" id="checkboxx20"></a>

Allows logic state input.

<figure><img src="/files/iSP7EhEH8AGF0e3lNxGE" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class CheckBox:
    def __init__(
        self,
        *,
        text: str = "",
        init_state: bool = False,
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def is_checked(self) -> bool: ...
```

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    ...
    def init(self):
        ...
        self.param['gray_mode'] = CheckBox(text=': Gray Mode')
    
    def run(self):
        ...
        flag_gray: bool = self.param['gray_mode'].is_checked
        ... 
```

### Button <a href="#button" id="button"></a>

Triggers an event in your script on mouse click. This component is also very useful for resource management for custom blocks in your scenario.

<figure><img src="/files/srvxBiJwZfkUqRKLu982" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
from collections.abc import Callable

class Button:
    def __init__(self, *, text: str = "", tool_tip: str = "", **kwargs) -> None: ...
    def set_clicked_callback(self, callback: Callable[[], None]) -> None: ...
```

{% hint style="info" %}
Use `set_clicked_callback(...)` inside `init()`.
{% endhint %}

<mark style="color:blue;">Example:</mark>

```python
...
class Example_Block(Block):
    ...
    file_path: str = ''
    def init(self):
        ...
        self.param['Choose File'] = Button(text= 'Choose File')
        self.param['Choose File'].set_clicked_callback(self.load_image)
    
    def load_image(self):
        path = QAFileDialog.getOpenFileName(caption='Load Image', 
                                        directory='C:/Images', 
                                        filter='Image Files (*.png *.jpg *.bmp)')
        self.file_path = self.register_resource('image-path', path)
        
    def run(self):
        image_path = self.get_resource('image-path')
    
```

Example above utilizes callbacks using [`register_resource`](/key-features/create-plugins-with-designer-window/coding-reference#blockregister_resourcename-str-path-str-str) and [`get_resource`](/key-features/create-plugins-with-designer-window/coding-reference#blockget_resourcename-str-str).

### Image <a href="#imagex20" id="imagex20"></a>

<figure><img src="/files/87hBzCq39KX5NBJ924Au" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
import numpy as np
import numpy.typing as npt

class Image:
    def __init__(
        self,
        *,
        fixed_width: int = 80,
        fixed_height: int = 80,
        tool_tip: str = "",
        serializable: bool = False,
        **kwargs,
    ) -> None: ...

    def update(self, img: npt.NDArray[np.uint8]) -> None: ...
```

<mark style="color:blue;">Example:</mark>

```python
...
class Example_Block(Block):
    def init(self):
        ...
        self.param['Result'] = Image(
            fixed_width=self.width - 40,
            fixed_height=self.height - 80,
        )

    def run(self):
        ...
        self.param['Result'].update(np.zeros((60, 60, 3), dtype=np.uint8))
        ...
```

### Table <a href="#table" id="table"></a>

Allows multiple items/modes to be selected at the same time.

<figure><img src="/files/Vk01JfBN7AG8G6hX8gKW" alt=""><figcaption></figcaption></figure>

```python
# pyi-style reference
class Table:
    def __init__(
        self,
        *,
        header_label: str = "table",
        items: list[str] | dict[str, bool] = ["item1", "item2", "item3"],
        tool_tip: str = "",
        **kwargs,
    ) -> None: ...

    @property
    def items(self) -> list[str]: ...

    @property
    def selected_items(self) -> list[str]: ...

    def set_items(self, items: list[str]) -> None: ...
```

<mark style="color:blue;">Example:</mark>

```python
...
class Example_Block(Block):
    def init(self):
        ...
        self.param['Detection List'] = Table(items=['Human', 'Cat', 'Dog'])

    def run(self):
        ...
        detection_list: list[str, ...] = self.param['Detection List'].selected_items
        ...
```

### Text Edit <a href="#text-edit" id="text-edit"></a>

Text Edit is a multi-line text area.

```python
# pyi-style reference
class TextEdit:
    def __init__(self, *, text: str = "", tool_tip: str = "", **kwargs) -> None: ...

    @property
    def value(self) -> str: ...

    @property
    def text(self) -> str: ...  # same as .value
```

<mark style="color:blue;">Example:</mark>

```python
class Example_Block(Block):
    def init(self):
        self.param['notes'] = TextEdit(tool_tip='Write notes here')

    def run(self):
        notes: str = self.param['notes'].value
```


# Coding Reference

This page explains how to write **Custom Blocks** for AugeLab Studio Designer.

Custom Blocks are plain Python classes that subclass `Block`, define sockets/components in `init()`, and process data in `run()`.

## Quick Start <a href="#quick-start" id="quick-start"></a>

* Start your script with `from studio.custom_block import *` (mandatory).
* Create a class that subclasses `Block`.
* Set `op_code` to the **exact class name**.
* Implement `init()` to configure sockets and UI components.
* Implement `run()` to read inputs and write outputs.
* Register the block at the bottom with `add_block(...)`.

{% hint style="info" %}
Designer Window extracts your block name from the first `class MyBlock(Block):` it can find, replaces tabs with 4 spaces, writes `<BlockName>.py`, then imports it to validate.
{% endhint %}

{% code title="Example Custom Block (copy/paste friendly)" %}

```python
from studio.custom_block import *

try:
    import numpy as np
except ImportError:
    np = None


class Example_Block(Block):
    op_code = "Example_Block"  # Must match the class name

    def init(self) -> None:
        self.width = 200
        self.height = 150
        self.tooltip = "Adds a constant to the input image."

        # Socket names become keys for self.input[...] and self.output[...]
        self.input_sockets = [SocketTypes.ImageAny("Input Image")]
        self.output_sockets = [SocketTypes.ImageAny("Output Image")]

        # Components are stored by name in self.param
        self.param["Increment"] = TextInput(
            text="1",
            place_holder="integer",
            tool_tip="Value added to every pixel.",
        )

    def run(self) -> None:
        if np is None:
            return

        img = self.input["Input Image"].data
        inc = int(self.param["Increment"].value)
        self.output["Output Image"].data = img + inc


add_block(Example_Block.op_code, Example_Block)
```

{% endcode %}

<figure><img src="/files/99A00bjsOtKy6VI2L56S" alt="Example Custom Block"><figcaption><p>Example Custom Block</p></figcaption></figure>

## Bootstrap Phase <a href="#bootstrap-phase" id="bootstrap-phase"></a>

### Mandatory Imports <a href="#mandatory-imports" id="mandatory-imports"></a>

Every Custom Block script must begin with `from studio.custom_block import *`.

It provides `Block`, `SocketTypes`, Designer components (TextInput, Slider, …), and `add_block`.

### Importing Community Modules <a href="#importing-community-modules" id="importing-community-modules"></a>

You may import third-party packages (NumPy, OpenCV, …) and use them inside `run()`.

If you install packages via the **Import Package Window**, make sure they work on all target platforms where your scenario will run.

{% hint style="warning" %}
All blocks shipped with AugeLab Studio are cross-platform compatible. Installing or importing community modules may reduce portability.
{% endhint %}

## Class Definition <a href="#class-definition" id="class-definition"></a>

The class name becomes the block name shown in the Custom Blocks list.

You can add helper methods and internal state as needed.

### Core Attributes & Methods <a href="#class-attributes-methods" id="class-attributes-methods"></a>

### `Block.op_code: str` <a href="#blockop_code-str" id="blockop_code-str"></a>

Unique identifier for your block.

For Designer-generated blocks, `op_code` should be the same as the class name.

{% hint style="info" %}
Keeping `op_code` and socket names stable helps Studio reconnect nodes when you update a block.
{% endhint %}

### `Block.tooltip: str` <a href="#blocktooltip-str" id="blocktooltip-str"></a>

Tooltip text shown when the user hovers over the block.

<figure><img src="/files/qtmqrIYqSdbRWqTngyp6" alt="Tooltip shown on custom block"><figcaption><p>Tooltip shown on custom block</p></figcaption></figure>

### `Block.init(self) -> None` <a href="#blockinitself" id="blockinitself"></a>

Called when the block is created (drag-drop), duplicated (copy/paste), or loaded from a scenario file.

Use this method to configure the block UI and sockets:

* Define `input_sockets` and `output_sockets`
* Create components in `self.param` (TextInput, DropDown, Slider, ...)
* Load and register resource paths if needed

### `Block.run(self) -> None` <a href="#blockrunself" id="blockrunself"></a>

Executed on every scenario step.

Read from `self.input`, write to `self.output`, and (optionally) update component state.

## Sockets & Data Flow <a href="#sockets-and-data" id="sockets-and-data"></a>

### `Block.input_sockets: list[SocketType]` <a href="#blockinput_sockets-listsockettype" id="blockinput_sockets-listsockettype"></a>

Defines which inputs the block accepts.

Each socket has a visible **name**; that same name becomes the key you use in `self.input[...]`.

<details>

<summary>Socket constructor</summary>

```python
SomeSocketClass(SocketTypes.BaseSocketClass):
    def __init__(self, name: str = "", multiple: bool = False):
        """ 
        name: text shown beside the socket graphics
        multiple: draw a horizontal line to indicate a list-type socket
        """
        ...
```

</details>

<details>

<summary>Available socket types</summary>

| Socket type                                   | Typical payload          |
| --------------------------------------------- | ------------------------ |
| `ImageAny`, `ImageRGB`, `ImageGray`           | `numpy.ndarray`          |
| `Mask`                                        | `numpy.ndarray`          |
| `Integer`                                     | `int`                    |
| `Number`                                      | `int` or `float`         |
| `Boolean`                                     | `bool`                   |
| `String`                                      | `str`                    |
| `Generic`                                     | any object               |
| `Shape`, `Contour`, `Range`, `Pixel`, `Point` | depends on upstream node |

</details>

{% hint style="info" %}
Exact payload types depend on runtime and upstream nodes. Commonly you'll see `numpy.ndarray` for images/masks and Python primitives (`int`, `float`, `bool`, `str`) for numeric/text sockets.
{% endhint %}

### `Block.output_sockets: list[SocketType]` <a href="#blockoutput_sockets-listsockettype" id="blockoutput_sockets-listsockettype"></a>

Same concept as inputs: define outputs and then write data by socket name using `self.output["Name"].data = ...`.

### `Block.input: dict[str, object]` <a href="#blockinput-dictstr-object" id="blockinput-dictstr-object"></a>

Runtime map of input socket names to objects that carry the payload in `.data`.

Each input also tracks connection state in `.is_connected`. If an input is not connected, `.data` is set to `None`.

```python
class Example_Block(Block):
    def init(self) -> None:
        self.input_sockets = [
            SocketTypes.ImageAny("Image"),
            SocketTypes.Number("Constant"),
        ]

    def run(self) -> None:
        image = self.input["Image"].data
        constant = self.input["Constant"].data
        ...
```

### `Block.output: dict[str, object]` <a href="#blockoutput-dictstr-object" id="blockoutput-dictstr-object"></a>

Runtime map of output socket names to objects that carry the payload in `.data`.

```python
class Example_Block(Block):
    def init(self) -> None:
        self.output_sockets = [
            SocketTypes.ImageAny("Result"),
            SocketTypes.Number("Detections"),
        ]

    def run(self) -> None:
        self.output["Result"].data = img_detections_drawn
        self.output["Detections"].data = n_detections
```

## Components (Block UI) <a href="#components" id="components"></a>

### `Block.param: dict[str, Component]` <a href="#blockparam-dictstr-component" id="blockparam-dictstr-component"></a>

Components are stored in `self.param` by a unique name you choose.

Use them to configure your block UI (text fields, sliders, buttons, images, tables, etc.).

* [Text Input](/key-features/create-plugins-with-designer-window/components#text-input)
* [Drop Down List](/key-features/create-plugins-with-designer-window/components#drop-down-list)
* [Label](/key-features/create-plugins-with-designer-window/components#label)
* [Slider](/key-features/create-plugins-with-designer-window/components#slider) / [Slider Labeled](/key-features/create-plugins-with-designer-window/components#slider-labeled)
* [Check Box](/key-features/create-plugins-with-designer-window/components#checkbox)
* [Button](/key-features/create-plugins-with-designer-window/components#button)
* [Image](/key-features/create-plugins-with-designer-window/components#image)
* [Table](/key-features/create-plugins-with-designer-window/components#table)

## Resource Paths <a href="#resources" id="resources"></a>

### `Block.register_resource(name: str = "", path: str = "") -> str` <a href="#blockregister_resourcename-str-path-str-str" id="blockregister_resourcename-str-path-str-str"></a>

Registers a file/folder path so it is stored with the scenario and can be relocated with the project. This avoids hard-coded absolute paths breaking when a scenario is moved to another computer.

| Argument | Meaning                            |
| -------- | ---------------------------------- |
| `name`   | Unique identifier for the resource |
| `path`   | File/folder path to register       |

Returns: `str` (the registered path)

To retrieve the stored path later, use `get_resource()`.

<details>

<summary>Example: let the user choose a file once</summary>

```python
from studio.custom_block import *


class LoadImage(Block):
    op_code = "LoadImage"

    def init(self):
        self.width = 220
        self.height = 180

        self.output_sockets = [SocketTypes.ImageAny("Image")]

        self.param["Pick File"] = Button(text="Pick File")
        self.param["Pick File"].set_clicked_callback(self._pick_file)

    def _pick_file(self):
        path = QAFileDialog.getOpenFileName(
            caption="Choose an image",
            directory="",
            filter="Image Files (*.png *.jpg *.jpeg *.bmp)",
        )
        if path:
            self.register_resource("image_path", path)

    def run(self):
        path = self.get_resource("image_path")
        if not path:
            return
        # TODO: load the image from 'path' and set self.output["Image"].data


add_block(LoadImage.op_code, LoadImage)
```

</details>

### `Block.get_resource(name: str = "") -> str` <a href="#blockget_resourcename-str-str" id="blockget_resourcename-str-str"></a>

Returns the registered path for a given resource name.

## File Dialogs <a href="#dialogs" id="dialogs"></a>

### `QAFileDialog` <a href="#qafiledialog" id="qafiledialog"></a>

Static helper for showing file/folder pickers. These dialogs return a path chosen by the user.

{% hint style="info" %}
Use file dialogs inside callbacks (e.g. a `Button` click), not inside `run()`.
{% endhint %}

#### `QAFileDialog.getOpenFileName(**kwargs)` <a href="#qafiledialoggetopenfilenamekwargs" id="qafiledialoggetopenfilenamekwargs"></a>

| Argument              | Meaning                                               |
| --------------------- | ----------------------------------------------------- |
| `caption: str = ""`   | Dialog title                                          |
| `directory: str = ""` | Start directory                                       |
| `filter: str = ""`    | File filter, e.g. `"Image Files (*.png *.jpg *.bmp)"` |

Returns: `str` (selected file path)

#### `QAFileDialog.getExistingDirectory(**kwargs)` <a href="#qafiledialoggetexistingdirectory" id="qafiledialoggetexistingdirectory"></a>

| Argument              | Meaning         |
| --------------------- | --------------- |
| `caption: str = ""`   | Dialog title    |
| `directory: str = ""` | Start directory |

Returns: `str` (selected directory path)

## Registration <a href="#registration" id="registration"></a>

### `add_block(My_Block.op_code, My_Block)` <a href="#add_blockmy_blockop_code-my_block" id="add_blockmy_blockop_code-my_block"></a>

Registers your block so it appears in the Designer. This is typically generated for you by the Designer Window. If you modify it, keep `op_code` and the class name consistent.


# Share Your Solutions with Community

Share your solutions with Plugin Manager

## First Look <a href="#first-look" id="first-look"></a>

<figure><img src="/files/zQxfO0Cc0hPSlwYyh9eB" alt="Plugin Window"><figcaption><p>Plugin Window (Upload / Download)</p></figcaption></figure>

The **Plugin Window** (Plugin Manager) is where you can:

* Download community solutions (blocks / models / scenarios / datasets)
* Upload your own solutions
* Publish for free (open source) or set a price

## Open the Plugin Window <a href="#accessing-the-plug-in-manager" id="accessing-the-plug-in-manager"></a>

* Menu: `Tools > Plugin Window`
* Shortcut: `Alt + P`
* Toolbar button: **Plugin Window**

## What Can Be Shared? <a href="#what-can-be-shared" id="what-can-be-shared"></a>

| Type     | Typical file         | What happens after download                                           |
| -------- | -------------------- | --------------------------------------------------------------------- |
| Block    | `.py` (custom block) | Auto-imported into **Custom Blocks** list when the download finishes. |
| Model    | archive (zip/rar/7z) | Downloaded into the Studio marketplace weights folder.                |
| Scenario | `.pmod`              | Downloaded into the Studio marketplace scenes folder.                 |
| Dataset  | archive (zip/rar/7z) | Downloaded into the Studio marketplace scenes folder.                 |

{% hint style="info" %}
The window has two tabs: **Upload Plugin** and **Download Plugin**.
{% endhint %}

## Upload a Plugin <a href="#uploading-a-custom-block" id="uploading-a-custom-block"></a>

<details>

<summary>Upload steps (Upload Plugin tab)</summary>

1. Open **Upload Plugin**.
2. Select **Plugin Type**:
   * `BLOCK`, `SCENARIO`, `MODEL`, or `DATASET`
3. Select **Plugin Class** (category) when available:
   * `Input`, `Basic Operations`, `Output`, `Image Processing`, `AI Applications`
4. Select the file:
   * **Block**: choose a block from the dropdown (from your local `custom_blocks` folder)
   * **Scenario/Model/Dataset**: click **Select Plugin File**
     * The picker expects an archive format (`.zip`, `.rar`, `.7z`).
5. Add a short description (how to use it, required dependencies, notes).
6. (Optional) Click **Select Screenshot Files** (max 3).
7. Choose your release style:
   * Check **Publish plugin as open source** to force price = `0`
   * Or set a price in **SET PLUGIN PRICE**
8. Click **Upload Selected Plugin** and monitor the progress bar/logs.

</details>

{% hint style="warning" %}
If you haven't already done so, make sure your AugeLab Studio account is associated with valid payment information. If payment information is missing, you will be prompted to update it. You can do this by visiting the [marketplace](https://marketplace.augelab.com/).
{% endhint %}

## Download Plugins <a href="#downloading-plugins" id="downloading-plugins"></a>

<details>

<summary>Download steps (Download Plugin tab)</summary>

1. Open **Download Plugin**.
2. Click **Refresh List** to fetch the latest items.
3. Select a plugin from the table to view its details.
4. Click **Download Selected Plugin** and wait for completion.

{% hint style="info" %}
If the downloaded item is a **Block**, Studio attempts to import/register it automatically after download. If it does not appear, restart AugeLab Studio and click the refresh button in the **Custom Blocks** list.
{% endhint %}

</details>

<details>

<summary>Troubleshooting</summary>

### Tips & Troubleshooting <a href="#tips-and-troubleshooting" id="tips-and-troubleshooting"></a>

| Issue                                           | What to try                                                                   |
| ----------------------------------------------- | ----------------------------------------------------------------------------- |
| Upload fails immediately                        | Make sure you selected a file and filled the description.                     |
| Block download succeeded but block doesn’t show | Restart Studio, then refresh the **Custom Blocks** list.                      |
| “Plugin already imported” warning               | Remove/backup the existing local file first, then restart and download again. |
| Download tab shows empty list                   | Check internet connection and click **Refresh List**.                         |

</details>


# Instal Python Packages

Install dependencies into the Studio Python environment.

The **Import Package Window** lets you add Python dependencies to the same environment that runs:

* Designer Window custom blocks (`custom_blocks/*.py`)
* other user scripts that execute inside AugeLab Studio

## First Look <a href="#first-look" id="first-look"></a>

<figure><img src="/files/UYo0tqrzg8TweVoOhjsc" alt="Import Package Window"><figcaption><p>Import Package Window</p></figcaption></figure>

## Open the Window <a href="#open-window" id="open-window"></a>

* Menu: `Tools > Import Package Window`

## Quick Cheat Sheet <a href="#quick-cheat-sheet" id="quick-cheat-sheet"></a>

| You want to…                                      | Use                                                   | Notes                                                              |
| ------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------ |
| Copy a single `.py` file into Studio              | `IMPORT SCRIPT TO STUDIO`                             | Good for small helpers (e.g., `utils.py`).                         |
| Copy an entire local folder (package) into Studio | `IMPORT PACKAGE TO STUDIO`                            | Choose a folder that contains Python modules.                      |
| Install from PyPI                                 | `INSTALL PACKAGE FROM PYPI` + `ADD PACKAGE TO STUDIO` | Internet required. You can enter multiple names (comma-separated). |
| Validate package name/version info                | `CHECK PACKAGE INFO`                                  | Prints metadata into the log panel.                                |
| Open the built-in help text                       | `Help` (top menu in the window)                       | Opens “Import Python Package Manual”.                              |

{% hint style="warning" %}
Installing community packages can reduce portability. If your scenario needs to run on another machine, install the same packages there too (same Python version + platform).
{% endhint %}

<details>

<summary>Install from PyPI</summary>

1. Open `Tools > Import Package Window`.
2. In **INSTALL PACKAGE FROM PYPI**, type one or more package names (comma-separated).
3. Click **CHECK PACKAGE INFO** to confirm the package exists and to review metadata.
4. Click **ADD PACKAGE TO STUDIO** and watch the log for progress/errors.

{% hint style="info" %}
Packages are installed into the Studio “pkgs” directory (the same environment used by Designer custom blocks).
{% endhint %}

</details>

<details>

<summary>Import a local script (.py)</summary>

1. Click **IMPORT SCRIPT TO STUDIO**.
2. Choose a `.py` file.
3. Confirm the success message.

Use this when you want to import a helper module and then do `import my_helper` from your custom block.

</details>

<details>

<summary>Import a local package (folder)</summary>

1. Click **IMPORT PACKAGE TO STUDIO**.
2. Choose a folder that contains your Python package/modules.
3. Confirm the success message (or read the error dialog).

{% hint style="info" %}
If you update the package later, re-import it (or remove the old copy first, then import again).
{% endhint %}

</details>

<details>

<summary>Troubleshooting</summary>

| Problem                          | What to try                                                                     |
| -------------------------------- | ------------------------------------------------------------------------------- |
| “Package not found”              | Double-check the name on `pypi.org` and retry **CHECK PACKAGE INFO**.           |
| Install fails mid-way            | Check your internet connection and read the log output.                         |
| A custom block crashes on import | Wrap optional imports in `try/except ImportError` and handle `None` in `run()`. |

</details>


# Headless Usage

This section covers running AugeLab Studio **without the desktop UI** via the public API class `studio.StudioScenario`.

Headless API gives huge performance boosts compared to the desktop UI, especially when running on servers without GPUs. It also allows easy integration into other Python apps/services. { % endhint %}

***

## Flows to Code

Your visiually programmed scenarios (`.pmod` files) can be run headlessly via Python code:

<figure><img src="/files/BkVglmr2mW48jSQVmiR5" alt="Flows to Code" width="400"><figcaption></figcaption></figure>

Flows to Code

```python
from studio import StudioScenario
scenario = StudioScenario(verification_code="YOUR_CODE_HERE")
scenario.load_scenario("your_scenario.pmod")

while True:
    try:
        scenario.run()
    except KeyboardInterrupt:
        break
scenario.cleanup()
```

Running the saved .pmod files with python scripts gives you several benefits:

* Implementing your own dashboards and UIs
* Integrating scenarios into larger applications
* Hooking into logging system in AugeLab runtime
* Running scenarios in Docker containers (CPU/CUDA)
* Batch processing multiple inputs
* Automating runs and scheduling

and many more!


# Installation

Installing headless AugeLab Studio varies depending on your operating system. Follow the instructions below for your specific platform.

## Windows

Using the AugeLab Studio installer manually sets up the headless runtime already and is the recommended way to install Headless Studio on Windows.

## Linux

Download the Linux installer script from [account.augelab.com](https://account.augelab.com). The installer supports Debian and Debian-based distributions, including Ubuntu. It creates an isolated virtual environment at `~/studio_venv`, installs the selected `studio` package profile, and creates desktop launchers when UI mode is selected.

{% hint style="warning" %}
The installer does not install NVIDIA drivers. For AI/GPU mode, install the NVIDIA driver on the host first. AugeLab Studio ships the CUDA runtimes it needs. On Jetson devices, use the JetPack/NVIDIA driver stack for your device.
{% endhint %}

### Full installation

Use this for the regular desktop + headless installation:

```bash
chmod +x installer.sh
./installer.sh --ui
```

This installs `studio[ui]`, creates `~/studio_venv`, adds the `augelab_studio` launcher, and creates desktop entries.

After installation, run:

```bash
augelab_studio
```

### Full installation with AI/GPU support

Install the NVIDIA driver first, then run:

```bash
chmod +x installer.sh
./installer.sh --ui --ai
```

This installs `studio[ui,gpu]`.

<details>

<summary>Headless only</summary>

Use this when you only need the Python API and do not need the desktop UI:

```bash
chmod +x installer.sh
./installer.sh --headless
```

This installs `studio`. UI launcher and desktop entries are not created.

</details>

<details>

<summary>Headless with AI/GPU support</summary>

Install the NVIDIA driver first, then run:

```bash
chmod +x installer.sh
./installer.sh --headless --ai
```

This installs `studio[gpu]`. UI launcher and desktop entries are not created.

</details>

<details>

<summary>Interactive install</summary>

Run the installer without mode flags to choose install mode interactively:

```bash
chmod +x installer.sh
./installer.sh
```

The installer asks whether to install headless or UI mode, then whether to include the AI/GPU extra.

</details>

<details>

<summary>Non-interactive install with environment variables</summary>

Use environment variables when running from automation:

```bash
INSTALL_UI=1 INSTALL_AI=0 ./installer.sh
INSTALL_UI=1 INSTALL_AI=1 ./installer.sh
INSTALL_UI=0 INSTALL_AI=0 ./installer.sh
INSTALL_UI=0 INSTALL_AI=1 ./installer.sh
```

`INSTALL_UI=1` selects UI mode. `INSTALL_AI=1` includes the AI/GPU extra.

</details>

<details>

<summary>Uninstall</summary>

```bash
./installer.sh --uninstall
```

This removes installer-created files:

* `~/studio_venv`
* `~/.local/bin/augelab_studio`
* AugeLab Studio desktop entries
* AugeLab Studio icons

Parent directories are removed only if empty.

</details>

<details>

<summary>Manual Python install</summary>

Use the installer when possible. If you need a manual Python environment, install into an isolated virtual environment instead of system Python:

```bash
apt-get update -y && \
    apt-get install -y --no-install-recommends \
    libdmtx0b \
    zbar-tools \
    build-essential \
    libgl1-mesa-glx \
    curl \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

curl -fsSL https://astral.sh/uv/install.sh | sh
uv venv -p 3.12 studio_venv
uv pip install --python studio_venv/bin/python \
    studio \
    --extra-index-url https://pyrepo.augelab.com \
    --extra-index-url https://download.pytorch.org/whl/cpu \
    --index-strategy unsafe-best-match
```

For AI/GPU manual install, install the NVIDIA driver first, then replace `studio` with `studio[gpu]`. AugeLab Studio ships the CUDA runtimes it needs.

</details>

## Docker with CPU/CUDA on x86\_64

For customer deployments, build the container image from a Dockerfile. AugeLab Studio is a closed-source application, so Docker installs the `studio` package from the AugeLab package index instead of building Studio from source.

Use [Docker Example](/key-features/headless/docker-example) as the canonical Docker guide. It includes:

* CPU Dockerfile
* GPU/CUDA Dockerfile
* `docker-compose.yml`
* `.env`
* scenario runner script
* mounted output folder

### CPU Docker

Use CPU Docker when your scenario does not need CUDA acceleration. The Docker example builds from `python:3.12-slim-bookworm` and installs `studio`.

### GPU/CUDA

AugeLab Studio supports CUDA acceleration for headless scenarios. GPU Docker requires compatible NVIDIA drivers and NVIDIA Container Toolkit on the host.

Use the GPU/CUDA Dockerfile in [Docker Example](/key-features/headless/docker-example). It builds from an NVIDIA CUDA runtime image and installs `studio[gpu]`.

## ARM64 (Raspberry Pi, Jetson Nano/AGX)

AugeLab Studio supports ARM64 architecture for headless usage on Debian-based devices such as Raspberry Pi and NVIDIA Jetson series.

Download the Linux ARM64 installer scripts from [account.augelab.com](https://account.augelab.com), then run the installer on the device.

### Full ARM64 installation

```bash
chmod +x installer.sh
./installer.sh --ui
```

This installs `studio[ui]`, creates `~/studio_venv`, adds the `augelab_studio` launcher, and creates desktop entries.

### Full ARM64 installation with AI/GPU support

Install the correct JetPack/NVIDIA driver stack for your device first.

```bash
chmod +x installer.sh
./installer.sh --ui --ai
```

This installs `studio[ui,gpu]`.

<details>

<summary>Headless ARM64 only</summary>

```bash
chmod +x installer.sh
./installer.sh --headless
```

This installs `studio`. UI launcher and desktop entries are not created.

</details>

<details>

<summary>Headless ARM64 with AI/GPU support</summary>

Install the correct JetPack/NVIDIA driver stack for your device first.

```bash
chmod +x installer.sh
./installer.sh --headless --ai
```

This installs `studio[gpu]`. UI launcher and desktop entries are not created.

</details>

<details>

<summary>ARM64 uninstall</summary>

```bash
./installer.sh --uninstall
```

</details>

{% hint style="info" %}
The ARM64 installer supports Debian and Debian-based distributions. If your device uses another Linux family, contact AugeLab for deployment guidance.
{% endhint %}

For Jetson GPU deployments, match the installer, JetPack/NVIDIA driver stack, and AugeLab package profile for your device.


# Quick Start

This section covers running AugeLab Studio **without the desktop UI** via the public API class `studio.StudioScenario`.

Whatever your platform is, AugeLab runtime is guaranteed to work the same way. { % endhint %}

## Quick Start

Now, we'll walk you through a quick example of running a headless scenario using Python code.

We are going to create a simple calculus scenario, and manage it via Python code.

Open AugeLab Studio desktop application, and create a new scenario with the following blocks:

<figure><img src="/files/neoySw7830HhT7Nd2XlQ" alt="Headless Quick Start" width="600"><figcaption></figcaption></figure>

Headless Calculus Example

Save the scenario as `calculus_example.pmod`.

Create a new Python script with the following code:

{% code title="headless\_calculus.py" %}

```python
from studio import StudioScenario
scenario = StudioScenario(verification_code="YOUR_CODE_HERE")
scenario.load_scenario("calculus_example.pmod")
for i in range(5):
    result = scenario.run((i,))
    print(f"Input: {i}, Output: {result[0]}")
scenario.cleanup()
```

{% endcode %}

> Replace `YOUR_CODE_HERE` with your actual verification code.

We need to use the python executable that has AugeLab Studio installed. If you are using virtual environments, make sure to activate the correct environment first. { % endhint %}

Run the script, and you should see the following output:

```bash
python headless_calculus.py
```

Running this should give you an output similar to:

```bash
... # studio initialization logs
1
2
3
4
5
```

## Transferring .pmod files

Transferring .pmod files are pretty straightforward. You can simply copy the .pmod files created in the desktop application to your headless environment.

If your .pmod file uses any external resources (e.g., images, models), before copying, make sure you have the following folder structure:

```
\project
    scenario.pmod
    \resources
        template.jpg
        model.weights
        model.names
        model.cfg
        ...
```

And saving the scenario before transferring the files. You can easily test this by moving your `project` folder to another location and running the scenario there again.

Some custom blocks may require additional resources or dependencies to be transferred or installed in the headless environment. Make sure to check the documentation for any custom blocks you are using. { % endhint % }

## Wrapping Up

You are now ready to run your own headless scenarios using AugeLab Studio! Explore the various blocks and functionalities available in the desktop application, and leverage them in your headless Python applications.

For further details on advanced features, please continue to read.


# Command Line Interface

Use the `studio` command line interface to verify a license and run saved `.pmod` scenarios without opening the desktop application.

The safest way to call the CLI is through the Python executable that has AugeLab Studio installed:

```bash
python -m studio --help
```

If your environment also exposes the `studio` console command, this works too:

```bash
studio --help
```

## Before You Start

You need:

* AugeLab Studio installed.
* A saved `.pmod` scenario.
* Your AugeLab verification code, unless the machine is already activated.
* The Python executable from the environment where `studio` is installed.

{% hint style="info" %}
Use absolute paths when running from services, scheduled tasks, Docker, or SSH sessions. This avoids running the wrong Python environment.
{% endhint %}

## Step 1: Locate Python

### Windows

If you installed with the AugeLab installer, the Python environment is usually:

```powershell
$py = "$env:USERPROFILE\studio_venv\Scripts\python.exe"
& $py -m studio --help
```

If you installed manually into a project virtual environment, point to that environment instead:

```powershell
$py = "C:\path\to\studio_venv\Scripts\python.exe"
& $py -m studio --help
```

If `studio` is on `PATH`, you can check it directly:

```powershell
studio --help
```

### Linux

If you installed with the Linux installer, the Python environment is usually:

```bash
PY="$HOME/studio_venv/bin/python"
"$PY" -m studio --help
```

If you installed manually into a project virtual environment, point to that environment instead:

```bash
PY="/path/to/studio_venv/bin/python"
"$PY" -m studio --help
```

If the virtual environment is already active:

```bash
python -m studio --help
```

### Docker

Inside the Docker examples, run the module form:

```bash
python -m studio --help
```

## Step 2: Verify License

Run this once per machine or container image environment:

```powershell
& $py -m studio verify "YOUR_VERIFICATION_CODE"
```

Linux:

```bash
"$PY" -m studio verify "YOUR_VERIFICATION_CODE"
```

Expected output:

```
Verification succeeded.
```

Do not hardcode real verification codes into shared scripts, Dockerfiles, or Git repositories. Use environment variables or secret storage when automating deployments.

## Step 3: Run Scenario

Windows:

```powershell
& $py -m studio run "C:\path\to\scenario.pmod"
```

Linux:

```bash
"$PY" -m studio run /path/to/scenario.pmod
```

The command keeps the scenario running until the scenario stops, fails, or you interrupt it with `Ctrl+C`.

{% hint style="warning" %}
Copy the complete project folder when a scenario uses external files such as images, models, calibration files, or custom block assets. Keep those files in the same relative locations used when the scenario was saved.
{% endhint %}

## Common Run Modes

Run a fixed number of completed steps:

```bash
"$PY" -m studio run scenario.pmod --step 10
```

Start with the web dashboard:

```bash
"$PY" -m studio run scenario.pmod --web --address 0.0.0.0 --port 8080
```

Use restart supervision for unattended runs:

```bash
"$PY" -m studio run scenario.pmod --on-fail restart --max-restarts 5 --restart-delay 3
```

Emit line-delimited JSON events for automation:

```bash
"$PY" -m studio run scenario.pmod --json
```

Change runtime log verbosity:

```bash
"$PY" -m studio run scenario.pmod --verbosity 20
```

Ignore scenario load errors only when you intentionally want to continue with missing optional resources:

```bash
"$PY" -m studio run scenario.pmod --ignore-errors
```

{% hint style="warning" %}
`--step` cannot be used together with `--web`.
{% endhint %}

## Command Reference

| Command                                                                  | Purpose                                               |
| ------------------------------------------------------------------------ | ----------------------------------------------------- |
| `python -m studio --help`                                                | Show top-level CLI help.                              |
| `python -m studio verify CODE`                                           | Register a verification code for the current machine. |
| `python -m studio run scenario.pmod`                                     | Run a saved scenario continuously.                    |
| `python -m studio run scenario.pmod --step 10`                           | Run a saved scenario for 10 completed steps.          |
| `python -m studio run scenario.pmod --web --address 0.0.0.0 --port 8080` | Run with the web dashboard.                           |
| `python -m studio run scenario.pmod --on-fail restart --max-restarts 5`  | Restart failed runs up to 5 times.                    |
| `python -m studio run scenario.pmod --json`                              | Emit JSON lifecycle and result records.               |

## Exit Codes

| Code  | Meaning                                          |
| ----- | ------------------------------------------------ |
| `0`   | Success.                                         |
| `2`   | Command usage error.                             |
| `3`   | License verification or license loading failure. |
| `4`   | Scenario load failure.                           |
| `5`   | Scenario runtime failure.                        |
| `6`   | Unexpected crash.                                |
| `7`   | Web dashboard startup failure.                   |
| `8`   | Restart retries exhausted.                       |
| `130` | Interrupted by user.                             |

## Troubleshooting

| Symptom                      | Fix                                                                          |
| ---------------------------- | ---------------------------------------------------------------------------- |
| `No module named studio`     | Use the Python executable from the Studio virtual environment.               |
| `studio` command not found   | Use `python -m studio` with the correct Python executable.                   |
| Scenario file not found      | Use an absolute `.pmod` path or run from the project folder.                 |
| License failure              | Run `studio verify` again and check the verification code.                   |
| Web dashboard does not start | Change `--port`, or check firewall and container port mapping.               |
| Scenario load failure        | Copy missing resources with the `.pmod`, or fix custom block/resource paths. |


# Web View

AugeLab Web View is a lightweight monitoring dashboard intended for **headless** runs.

<figure><img src="/files/JNDssQiaHnDr59MY4Vf0" alt="Headless Web View" width="600"><figcaption></figcaption></figure>

Headless Web View

It shows:

* A live image preview (from your scenario output)
* A rolling log panel (from block/scenario logs)

***

## Start the Web View

All you need to do when you are using web-view, is to use a `Subsystem Out` block to connect to your final image:

<figure><img src="/files/ZVMDd6d3ND7d5SZaCEVX" alt="Web View Link" width="600"><figcaption></figcaption></figure>

Web View Scenario

Then, write a small python script:

{% code title="headless\_webview\.py" %}

```python
from studio import StudioScenario

scenario = StudioScenario()
scenario.load_scenario(r"PATH_TO_SCENARIO.pmod")
scenario.run_server(
    host="127.0.0.1",
    port=8080,
    header="Scenario Server",
)
```

{% endcode %}

Then open:

* `http://127.0.0.1:8080`

<figure><img src="/files/BYP9USKibSUufbSphOsu" alt="Web View Link" width="400"><figcaption></figcaption></figure>

Web View Running

***

## What the dashboard expects from your scenario

### Output image selection

The Web View displays the **first image-like output** produced by the scenario. If your scenario doesn’t output an image, the preview will stay blank.

### Image format

* The preview expects an image output.
* If the array looks like a BGR image (`H x W x 3`), it is channel-swapped to RGB before display.

{% hint style="info" %}
If your scenario’s “first output” is not an image, the dashboard will likely stay blank. In that case, adjust the scenario so the first output group emits an image.
{% endhint %}

***

## Logs

The dashboard shows recent runtime logs.

***

## Operational notes / limitations

* The scenario is executed in an infinite loop in a background thread.
* Stop the server with Ctrl+C.


# Docker Example

This page shows a reproducible Docker setup for running an AugeLab Studio `.pmod` scenario headlessly.

The image is built locally from a Dockerfile. Customers do not need the AugeLab Studio source code. The container installs the closed-source `studio` package from the AugeLab package index during image build.

## Prerequisites

* Docker Desktop or Docker Engine with Docker Compose.
* Access to the AugeLab package index.
* An AugeLab verification code.
* A `.pmod` scenario that can run headlessly.
* For GPU/CUDA: NVIDIA Container Toolkit installed on the host.

## Quick path

1. Create the folder layout below.
2. Put your `.pmod` file in `app/`.
3. Add your verification code to `.env`.
4. Use the CPU or GPU Dockerfile.
5. Run `docker compose up --build`.
6. Check `output_on_host`.

## Project layout

```
augelab-docker-example/
  .env
  docker-compose.yml
  docker-compose.cuda.yml
  output_on_host/
  app/
    Dockerfile
    Dockerfile.cuda
    run_scenario.py
    your_scenario.pmod
```

Replace `your_scenario.pmod` with your own scenario file.

## Environment file

Create `.env` next to `docker-compose.yml`:

<details>

<summary>.env</summary>

```env
AUGELAB_VERIFICATION_CODE=PASTE_YOUR_VERIFICATION_CODE_HERE
SCENARIO_PATH=your_scenario.pmod
```

</details>

Do not commit `.env` if it contains a real verification code.

## Scenario runner

Create `app/run_scenario.py`:

<details>

<summary>app/run_scenario.py</summary>

```python
import os
from pathlib import Path

from studio import StudioScenario


OUTPUT_DIR = Path("/app/app_output")
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

verification_code = os.environ.get("AUGELAB_VERIFICATION_CODE")
if not verification_code:
    raise RuntimeError("AUGELAB_VERIFICATION_CODE is not set.")

scenario_path = os.environ.get("SCENARIO_PATH", "your_scenario.pmod")
if not Path(scenario_path).is_file():
    raise FileNotFoundError(f"Scenario file not found: {scenario_path}")

scenario = StudioScenario(verification_code=verification_code)
scenario.enable_logging_stdout()
scenario.load_scenario(scenario_path)

try:
    print("Scenario result:", scenario.run())
finally:
    scenario.cleanup()
```

</details>

If your scenario writes files, configure the scenario to write them under `/app/app_output`.

## CPU Dockerfile

Use this for regular headless CPU runs.

<details>

<summary>app/Dockerfile</summary>

```dockerfile
FROM python:3.12-slim-bookworm

WORKDIR /app

RUN apt-get update -y && \
    apt-get install -y --no-install-recommends \
    libdmtx0b \
    zbar-tools \
    build-essential \
    libgl1-mesa-glx \
    curl \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

ADD https://astral.sh/uv/0.9.17/install.sh /uv-installer.sh
RUN sh /uv-installer.sh && rm /uv-installer.sh

ENV PATH="/root/.local/bin/:$PATH"

RUN uv pip install studio --system \
    --extra-index-url https://pyrepo.augelab.com \
    --extra-index-url https://download.pytorch.org/whl/cpu \
    --index-strategy unsafe-best-match

RUN python -c "from studio import StudioScenario; print('studio import ok')"

COPY run_scenario.py .
COPY your_scenario.pmod .

CMD ["python", "run_scenario.py"]
```

</details>

## GPU/CUDA Dockerfile

Use this when your scenario needs CUDA acceleration. The host must have compatible NVIDIA drivers and NVIDIA Container Toolkit.

<details>

<summary>app/Dockerfile.cuda</summary>

```dockerfile
FROM nvidia/cuda:12.8.0-cudnn-runtime-ubuntu22.04

WORKDIR /app

RUN apt-get update -y && \
    apt-get install -y --no-install-recommends \
    python3 \
    python3-venv \
    python3-pip \
    libdmtx0b \
    zbar-tools \
    build-essential \
    libgl1-mesa-glx \
    curl \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

ADD https://astral.sh/uv/0.9.17/install.sh /uv-installer.sh
RUN sh /uv-installer.sh && rm /uv-installer.sh

ENV PATH="/root/.local/bin/:$PATH"
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility

RUN uv pip install "studio[gpu]" --system \
    --extra-index-url https://pyrepo.augelab.com \
    --index-strategy unsafe-best-match

RUN python3 -c "from studio import StudioScenario; print('studio gpu import ok')"

COPY run_scenario.py .
COPY your_scenario.pmod .

CMD ["python3", "run_scenario.py"]
```

</details>

## Compose file

Use this for the CPU Dockerfile:

<details>

<summary>docker-compose.yml</summary>

```yaml
services:
  augelab-headless:
    build:
      context: ./app
      dockerfile: Dockerfile
    env_file:
      - .env
    volumes:
      - ./output_on_host:/app/app_output
```

</details>

This mounts `./output_on_host` from your project folder into the container as `/app/app_output`.

{% hint style="warning" %}
On Windows, use forward slashes in absolute Docker mount paths, for example `C:/work/augelab_output:/app/app_output`. Relative mounts like `./output_on_host:/app/app_output` are usually easier to share across machines.
{% endhint %}

## GPU/CUDA compose file

Use this for `Dockerfile.cuda`:

<details>

<summary>docker-compose.cuda.yml</summary>

```yaml
services:
  augelab-headless-gpu:
    build:
      context: ./app
      dockerfile: Dockerfile.cuda
    env_file:
      - .env
    volumes:
      - ./output_on_host:/app/app_output
    gpus: all
```

</details>

## Build and run

CPU:

```bash
docker compose up --build
```

GPU/CUDA:

```bash
docker compose -f docker-compose.cuda.yml up --build
```

You should see AugeLab Studio logs and the scenario result in the terminal.

## Verify output

Check `output_on_host` on your host machine. Files written by the scenario to `/app/app_output` should appear there.

## Troubleshooting

| Problem                       | Check                                                                                                                       |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `studio` install fails        | Confirm network access and AugeLab package index access.                                                                    |
| License/activation fails      | Confirm `AUGELAB_VERIFICATION_CODE` is set and the container has internet access.                                           |
| Scenario file not found       | Confirm `SCENARIO_PATH` matches the `.pmod` copied by the Dockerfile.                                                       |
| Output folder empty           | Confirm the `.pmod` writes to `/app/app_output`.                                                                            |
| Windows volume does not mount | Use forward slashes and confirm Docker Desktop can access the drive.                                                        |
| GPU container cannot see GPU  | Install NVIDIA Container Toolkit and test with `docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi`. |


# Scripting Reference

This page documents the **public headless scripting API** exposed by the `studio` Python package.

It focuses on these public symbols:

* `StudioScenario`
* `install_logging_event`
* `enable_logging_stdout`
* `disable_logging_stdout`

If you haven't run a scenario headlessly before, read the Headless section first and then come back here for API details.

***

## Imports

All of the APIs documented here are exported from the top-level `studio` package:

```python
from studio import (
	StudioScenario,
	install_logging_event,
	enable_logging_stdout,
	disable_logging_stdout,
)
```

***

## Type Stubs

The public headless surface can be found here:

```python
from __future__ import annotations

from typing import Any, Callable, Optional


def enable_logging_stdout() -> None: ...
def disable_logging_stdout() -> None: ...
def install_logging_event(event: Callable[[str], None]) -> None: ...


class StudioScenario:
	def __init__(self, *, verification_code: str = "") -> None: ...

	def load_scenario(self, path: str) -> Optional[StudioScenario]: ...
	def load_scenario_raw(self, content: str) -> bool: ...
	def save_scenario(self, path: str) -> bool: ...
	def clear_scenario(self) -> bool: ...
	def disable_load_errors(self) -> StudioScenario: ...
	def load_custom_nodes(self, path: str) -> bool: ...

	def get_ports(
		self,
		input_op_title: str = "Subsystem In",
		output_op_title: str = "Subsystem Out",
	) -> dict[str, list[str]]: ...
	def get_name(self) -> str: ...

	def run(self, args: tuple[Any, ...] = ()) -> tuple[list[Any], ...]: ...
	def run_server(
		self,
		inputs: tuple[Any, ...] = (),
		host: str = "127.0.0.1",
		port: int = 8080,
		display_controls: bool = False,
		header: str = "Scenario Server",
	) -> None: ...
	def cleanup(self) -> bool: ...

	def enable_logging_stdout(self) -> StudioScenario: ...
	def disable_logging_stdout(self) -> StudioScenario: ...
	def install_logging_hook(self, hook: Callable[[str, int], None], level: int = 0) -> StudioScenario: ...

	def installStartEvent(self, event: Callable[..., None]) -> StudioScenario: ...
	def installEndEvent(self, event: Callable[..., None]) -> StudioScenario: ...
	def installStepStartEvent(self, event: Callable[..., None]) -> StudioScenario: ...
	def installStepEndEvent(self, event: Callable[..., None]) -> StudioScenario: ...
```

***

## `StudioScenario`

`StudioScenario` is the main entry point for **loading and running** a `.pmod` file without the desktop UI.

### Constructor

```python
scenario = StudioScenario(verification_code="...")
```

#### `verification_code`

This value is used to configure licensing for headless execution.

Common patterns:

* **Local machine / server:** pass the verification code string.
* **Docker / CI:** mount a license file into the container and pass its **path** as `verification_code`.

For non-interactive environments (Docker/CI/services), always pass `verification_code` explicitly. { % endhint %}

***

### Typical lifecycle

In most scripts you will:

1. Create the scenario (`StudioScenario(...)`)
2. Load a `.pmod` (`load_scenario(...)`)
3. Run it (`run(...)`)
4. Cleanup (`cleanup()`)

```python
import os

from studio import StudioScenario, enable_logging_stdout


def main() -> None:
	# Optional: prints runtime logs to stdout (useful when debugging).
	enable_logging_stdout()

	verification_code = os.environ.get("AUGELAB_VERIFICATION_CODE", "")
	scenario = StudioScenario(verification_code=verification_code)

	try:
		loaded = scenario.load_scenario("your_scenario.pmod")
		if loaded is None:
			raise FileNotFoundError("Could not load .pmod file")

		# If your scenario has N input ports, you must pass exactly N args.
		result = scenario.run(args=())
		print("Scenario result:", result)
	finally:
		scenario.cleanup()


if __name__ == "__main__":
	main()
```

`cleanup()` is important in long-running processes (services, batch jobs) to release memory and reset runtime state. { % endhint %}

***

### Loading a `.pmod`

```python
scenario.load_scenario("path/to/scenario.pmod")
```

* Returns `StudioScenario` on success.
* Returns `None` if the path does not exist.

If your scenario references external resources, keep the same directory structure used on desktop (see the “Transferring .pmod files” section).

***

### Running a scenario

```python
outputs = scenario.run(args=(... ,))
```

The runtime validates that the number of arguments matches the number of scenario inputs.

If your `.pmod` has:

* 0 inputs: call `scenario.run()` or `scenario.run(())`
* 1 input: call `scenario.run((value,))`
* 2 inputs: call `scenario.run((value1, value2))`

`args` must be a tuple. For a single input, remember the trailing comma: `(value,)`. { % endhint %}

***

## More `StudioScenario` Methods

This section covers additional public methods you can use for automation and debugging.

### Loading without failing on missing resources

If you are moving scenarios between machines/containers and some resources may be unavailable, you can choose to load more permissively:

```python
from studio import StudioScenario

scenario = StudioScenario(verification_code="YOUR_CODE")
scenario.disable_load_errors()
scenario.load_scenario("scenario.pmod")
```

### Load / save

```python
scenario.load_scenario_raw(content="...")
scenario.save_scenario("out.pmod")
scenario.clear_scenario()
```

### Custom nodes

Load custom node Python modules from a folder:

```python
scenario.load_custom_nodes("path/to/custom_nodes")
```

This dynamically imports Python files. Only load trusted code. { % endhint %}

### Scenario name

```python
name = scenario.get_name()
```

### Ports (scenario inputs/outputs)

You can query the input/output port names:

```python
ports = scenario.get_ports()
print("inputs:", ports["inputs"])
print("outputs:", ports["outputs"])
```

### Run a web server (optional)

If you want to expose a scenario runner over HTTP:

```python
scenario.run_server(host="0.0.0.0", port=8080)
```

### Events

Install lifecycle callbacks around scenario execution:

```python
def on_start() -> None:
	print("Scenario started")


scenario.installStartEvent(on_start)
```

***

## Logging Helpers

AugeLab Studio has a runtime logger used by blocks and scenarios. In headless mode you typically want one of these:

* print logs to stdout while developing
* forward logs into your own logging system

These helpers are **global** (they affect the runtime logger used by your headless scenario).

***

### `enable_logging_stdout()`

Enables “friendly” runtime logging to stdout.

Use this when you want to see block/scenario logs in the console:

```python
from studio import enable_logging_stdout

enable_logging_stdout()
```

***

### `disable_logging_stdout()`

Disables runtime logging to stdout.

Useful when:

* you are using `install_logging_event(...)` and want to avoid duplicated logs
* you want clean output (only your own `print(...)` / logger output)

```python
from studio import disable_logging_stdout

disable_logging_stdout()
```

***

### `install_logging_event(event: Callable[[str], None])`

Installs a callback that receives runtime log messages.

This is the easiest way to integrate AugeLab runtime logs into your own logging framework.

```python
import logging

from studio import install_logging_event, disable_logging_stdout

logger = logging.getLogger("augelab")


def forward_to_python_logging(msg: str) -> None:
	logger.info(msg)


disable_logging_stdout()
install_logging_event(forward_to_python_logging)
```

### `StudioScenario.install_logging_hook(hook, level=0)`

If you prefer per-scenario hook installation (with filtering by level), use:

```python
def hook(msg: str, level: int) -> None:
	print(level, msg)


scenario.install_logging_hook(hook, level=20)
```

Keep your callback fast. If you need to do heavy work (I/O, network), consider buffering messages and processing them in another thread/process. { % endhint %}

***

## Recommended Patterns

### 1) Environment variable for licensing

For scripts that run in different environments (local vs CI vs Docker), passing `verification_code` via an environment variable keeps code portable:

```python
import os
from studio import StudioScenario

scenario = StudioScenario(verification_code=os.environ["AUGELAB_VERIFICATION_CODE"])
```

### 2) Always cleanup with `try/finally`

```python
from studio import StudioScenario

scenario = StudioScenario(verification_code="YOUR_CODE")
try:
	scenario.load_scenario("scenario.pmod")
	scenario.run(())
finally:
	scenario.cleanup()
```


# Camera Usage

Connect USB, IP, and industrial cameras to AugeLab Studio.

AugeLab Studio supports multiple camera types through dedicated blocks. This section helps you pick the right camera block and get a stable live image stream first (then you can start your vision pipeline).

## Quick start (choose your camera type)

* **USB webcam / UVC camera** → use [USB cameras](/devices-and-communications/camera-usage/usb)
* **IP camera with RTSP** → use [IP (RTSP)](/devices-and-communications/camera-usage/ip)
* **IP camera with ONVIF** → use [IP (ONVIF)](/devices-and-communications/camera-usage/ip)
* **Industrial cameras with vendor SDK** → see:
  * [Basler](/devices-and-communications/camera-usage/basler)
  * [HikRobot](/devices-and-communications/camera-usage/hikrobot)
  * [iRayple](/devices-and-communications/camera-usage/irayple)

{% hint style="info" %}
If you can preview your camera in the vendor tool (pylon Viewer, HikRobot MVS, iRayple platform) but not in AugeLab Studio, it is usually a driver, IP, or firewall issue—not a vision pipeline issue.
{% endhint %}

## Before you start (recommended checklist)

1. **Power + cable**: Confirm the camera is powered and connected (USB / Ethernet / PoE).
2. **Vendor viewer**: Verify you can see a live image in the vendor software (or in VLC for RTSP).
3. **One app at a time**: Close other software that may be holding the camera.
4. **Network** (GigE/IP): Put the camera and PC on the same subnet; avoid Wi‑Fi for streaming.

<details>

<summary>How to find camera blocks in AugeLab Studio</summary>

Most camera blocks are under the blocks library in **Input/Output → Image Inputs**. If a camera type is delivered as a plugin, install it from the **Plugin Window**.

</details>

<details>

<summary>Need help with a camera model?</summary>

If you think your camera is not supported (or you are unsure which block to use), contact `info@augelab.com` with:

* Camera brand + exact model
* Connection type (USB / GigE / RTSP / ONVIF)
* Windows version and AugeLab Studio version
* Screenshot of the vendor viewer (or RTSP URL format, without passwords)

</details>


# Basler

Set up Basler (GigE/USB3) cameras with pylon and AugeLab Studio.

Basler cameras typically connect via **GigE** or **USB3**. The recommended approach is always the same: verify the camera in **Basler pylon Viewer** first, then connect it in AugeLab Studio.

## Quick start

1. Install **Basler pylon** on the PC.
2. Verify live view in **pylon Viewer**.
3. (GigE) Put the camera and PC on the same subnet and set a stable IP.
4. Open AugeLab Studio and use the **Basler Camera** block.

## Requirements

* Basler camera (GigE or USB3)
* Power (external power supply or PoE, depending on model)
* For GigE: Ethernet cable and a GigE-capable network adapter (a switch is optional)

## Setup (GigE recommended steps)

1. **Connect and power** the camera.
2. Install Basler **pylon** (includes pylon Viewer): `https://www.baslerweb.com/en/downloads/software-downloads/`
3. Open **pylon Viewer** and confirm you can:
   * See the camera in the device list
   * Start acquisition and view a live image
4. In pylon Viewer (or the pylon IP tool), set a **static IP** that matches your PC subnet (example: PC `192.168.0.10`, camera `192.168.0.20`).
5. If AugeLab Studio asks for **Destination Address**, enter the computer's Ethernet/IP address, not the camera IP. This is usually the PC address on the same network, for example `192.168.0.10`.
6. Open AugeLab Studio and add the **Basler Camera** block. Select your camera and start acquisition.

<details>

<summary>Installing Python dependencies (only if needed)</summary>

Most AugeLab Studio installations already include the Basler Python bindings (`pypylon`). If your Basler block reports a missing dependency, install it via the **Package Window**:

* [Install Python Packages](/key-features/instal-custom-packages-with-package-window)

</details>

<details>

<summary>Troubleshooting</summary>

* **Camera visible in pylon Viewer but not in Studio**: close pylon Viewer; only one app can usually hold the camera at a time.
* **Camera not visible on GigE**: check that PC and camera are on the same subnet; disable VPN adapters and verify Windows firewall rules. If needed, set **Destination Address** to the computer's Ethernet/IP address.
* **Drops / stutter**: use a direct connection or a quality switch; prefer a dedicated NIC and avoid congested networks.

</details>

If you still can’t stream a stable image, contact `info@augelab.com` with your camera model and a screenshot from pylon Viewer.


# HikRobot

Set up HikRobot industrial cameras (GigE) with MVS and AugeLab Studio.

HikRobot industrial cameras require the **HikRobot MVS (Machine Vision Software)** installation on your PC. After that, you can connect from AugeLab Studio using the HikRobot camera block.

## Quick start

1. Install **HikRobot MVS**.
2. Verify live view in the MVS tool.
3. Set a stable IP (GigE) and confirm the camera is reachable.
4. Start streaming in AugeLab Studio using the HikRobot camera block.

## Requirements

* HikRobot camera (GigE or USB, depending on model)
* Power (external power supply or PoE, depending on model)
* For GigE: Ethernet cable and a GigE-capable network adapter (a switch is optional)

## Setup (GigE cameras)

1. **Connect and power** the camera.
2. Download and install **MVS**: `https://www.hikrobotics.com/en/machinevision/service/download/`
3. Open the MVS application and confirm you can start acquisition and see a live image.
4. Set a **static IP** for the camera that matches your PC subnet.
5. Open AugeLab Studio and add the **HikRobot Camera** block. Type your camera's IP address and start acquisition.

![HikCamera Usage](/files/ph4u7JpF8wr87EX74sBq)

<details>

<summary>Troubleshooting</summary>

* **Camera visible in MVS but not in Studio**: close MVS; only one app can usually connect to the camera at a time.
* **Can’t find the camera on GigE**: confirm the camera and PC are on the same subnet; try a direct Ethernet connection.

</details>


# Irayple

Set up iRayple industrial cameras with the iRayple machine vision platform.

iRayple industrial cameras require the vendor’s **Machine Vision Platform** installation. Once the driver stack is installed and the camera works in the vendor viewer, you can connect it in AugeLab Studio.

## Quick start

1. Install the iRayple **Machine Vision Platform**.
2. Verify live view in the vendor tool.
3. (GigE) Set a stable IP and confirm the camera is reachable.
4. Open AugeLab Studio and use the iRayple camera block/plugin.

## Requirements

* iRayple camera (GigE/USB, depending on model)
* Power (external power supply or PoE, depending on model)
* For GigE: Ethernet cable and a GigE-capable network adapter (a switch is optional)

## Install the vendor platform

Download and install the iRayple machine vision platform from the vendor download center:

* `https://www.irayple.com/en/serviceSupport/downloadCenter/18?p=17`

After installation, confirm you can start acquisition and see a live image in the iRayple software tools.

## Connect in AugeLab Studio

1. Close the iRayple viewer/tools (so the camera is not locked by another app).
2. Open AugeLab Studio and add the **iRayple Camera** block (or iRayple plugin blocks).
3. Select your camera and start acquisition.

<details>

<summary>If you don’t see an iRayple block</summary>

iRayple support may be delivered as a plugin depending on your AugeLab Studio installation.

* Check the **Plugin Window** for iRayple-related plugins.
* If you are unsure what to install, contact `info@augelab.com` with your camera model and connection type.

</details>

<details>

<summary>Troubleshooting</summary>

* **Camera visible in vendor tool but not in Studio**: close the vendor tool and retry.
* **GigE camera not discovered**: ensure PC and camera are on the same subnet; try a direct Ethernet connection.
* **Unstable stream**: reduce resolution/FPS; use a dedicated NIC; avoid Wi‑Fi and congested networks.

</details>


# USB

Use USB webcams and UVC cameras in AugeLab Studio.

USB cameras are the simplest way to get a live image into AugeLab Studio. Most webcams and many industrial USB cameras that expose a **UVC** interface work out of the box.

## Quick start

1. Plug in the camera (prefer a direct USB port, not a hub).
2. Close other apps that may be using the camera (Teams/Zoom/browser tabs).
3. In AugeLab Studio, add **Camera USB** and click **Scan Connected USB Cameras**.
4. Select the camera and activate it.

![USB Camera Usage](/files/2z694b75eQcfUjqwsYey)

## Which USB block should I use?

* **Camera USB** (recommended starting point): simple UI scanning and selection.\
  Block reference: [Camera USB](/function-blocks/blocks-reference/input-output/image-inputs/camera-usb)
* **Camera USB Vidgear**: alternative capture backend if you have compatibility issues.\
  Block reference: [Camera USB Vidgear](/function-blocks/blocks-reference/input-output/image-inputs/camera-usb-vidgear)
* **Camera USB External**: advanced control via inputs (index, resolution, exposure).\
  Block reference: [Camera USB External](/function-blocks/blocks-reference/input-output/image-inputs/camera-usb-external)

<details>

<summary>Tips for stable FPS</summary>

* Start with a lower resolution (e.g., 640×480) and increase gradually.
* Avoid USB hubs and long cables; use USB 3 ports when available.
* If you need multiple USB cameras, prefer separate USB controllers (different ports can still share a controller).

</details>

<details>

<summary>Troubleshooting</summary>

* **No cameras found**: verify Windows can see the camera (Device Manager); reconnect the camera; try a different USB port.
* **Black/blank frames**: release and re-activate the block; close other apps using the camera.
* **Wrong camera picked**: scan again and try a different camera index/selection.

</details>


# IP

Connect RTSP or ONVIF IP cameras to AugeLab Studio.

Use IP cameras when you need longer cable runs, network distribution, or multiple camera setups. AugeLab Studio supports both **RTSP** (fast, direct stream) and **ONVIF** (device discovery + profiles; typically lower FPS).

{% hint style="info" %}
If you can open the stream in VLC (RTSP) or in the vendor tool (ONVIF), you are very close—most remaining issues are credentials, URL format, or subnet/firewall settings.
{% endhint %}

<details>

<summary>RTSP (recommended for higher FPS)</summary>

Best when you need a higher frame rate and you know the RTSP URL (typical for up to a few cameras per PC, depending on resolution and codec).

**Setup**

1. Connect the camera to your switch/router (Ethernet) and power it on (PoE or adapter).
2. Set/confirm the camera **IP address** (check the vendor tool or camera web UI).
3. Determine the **RTSP URL** (camera manual/vendor UI). Common formats look like:
   * `rtsp://<user>:<pass>@<ip>:554/stream1`
   * `rtsp://<ip>:554/h264/ch1/main/av_stream`
4. In AugeLab Studio, add the **Camera IP** block and paste the RTSP URL.
   * Block reference: [Camera IP](/function-blocks/blocks-reference/input-output/image-inputs/camera-ip)

![RTSP Camera Usage](/files/VBEEl72MoBdQghQ6r3ko)

</details>

<details>

<summary>ONVIF (recommended for discovery and larger setups)</summary>

Use ONVIF if the camera supports it and you prefer device discovery / profiles. ONVIF streams may not reach the same FPS as direct RTSP (depends on camera and network).

**Setup**

1. Connect and power the camera (Ethernet + PoE/adapter).
2. Enable **ONVIF** in the camera settings (vendor UI) and set an ONVIF user if required.
3. In AugeLab Studio, add the **Camera IP (ONVIF)** block and enter:
   * Camera IP
   * Username/password
4. Start acquisition.

![ONVIF Camera Usage](/files/FquaPVywoRml24BNoCgD)

</details>

<details>

<summary>Troubleshooting (RTSP/ONVIF)</summary>

* **No connection**: verify PC and camera are on the same subnet; avoid VPN adapters; check Windows firewall.
* **RTSP connects but no image**: try a different stream path (main/sub stream) or switch codec (H.264/H.265) in the camera UI.
* **Stutter**: lower resolution/FPS/bitrate on the camera; prefer wired Ethernet; avoid Wi‑Fi.

</details>


# Others

How to use cameras not listed in this section.

If your camera is not listed in this section, it may still work with AugeLab Studio—most cameras fall into one of the categories below.

## Quick decision guide

| What your camera provides            | Recommended in AugeLab Studio                                     |
| ------------------------------------ | ----------------------------------------------------------------- |
| Appears as a Windows webcam (UVC)    | [USB Cameras](/devices-and-communications/camera-usage/usb)       |
| An RTSP URL (H.264/H.265 stream)     | [IP Cameras (RTSP)](/devices-and-communications/camera-usage/ip)  |
| ONVIF discovery / profiles           | [IP Cameras (ONVIF)](/devices-and-communications/camera-usage/ip) |
| Vendor SDK / viewer app (industrial) | Vendor-specific block/plugin (see Basler/HikRobot/iRayple pages)  |

<details>

<summary>Common signs your camera is UVC</summary>

* It shows up in Windows camera settings and works in the built-in Camera app.
* It works in video conferencing apps without installing vendor SDKs.

</details>

<details>

<summary>Common signs your camera is RTSP</summary>

* The camera has a web UI with “Stream” settings (codec, bitrate, main/sub stream).
* The vendor documentation includes an RTSP URL pattern.
* You can preview it in VLC using “Open Network Stream”.

</details>

## Need us to confirm support?

Email `info@augelab.com` and include:

* Brand + exact model number
* Connection type (USB / GigE / RTSP / ONVIF)
* Vendor SDK/viewer name (if any) + version
* What you tried and what you see (screenshots are great)


# Communication Protocols

AugeLab Studio supports several communication protocols to enable seamless integration and data exchange with various systems and devices.

Each communication block resides under the **Blocks➡️ Input/Output ➡️ Communication** section.

Below are the details on how to use each protocol:

<details>

<summary>REST API</summary>

REST is a web service protocol that uses HTTP requests to GET, POST data.

#### Usage <a href="#usage" id="usage"></a>

* **GET Request Block**: Use this block to retrieve data from a server. Configure the URL and other necessary parameters.
* **POST Request Block**: Use this block to send data to a server. Configure the URL, headers, and payload as needed.

</details>

<details>

<summary>OPC</summary>

OPC (OLE for Process Control) is a series of standards and specifications for industrial telecommunication. It is used for communication between devices and control applications in industrial environments.

#### Usage <a href="#usage" id="usage"></a>

* **OPC Client Block**: Use this block to connect to an OPC server. Configure the server address and other necessary parameters.
* **OPC Read/Write Blocks**: Use these blocks to read data from or write data to the OPC server.

Ensure the OPC server is correctly configured and accessible from AugeLab Studio.

</details>

<details>

<summary>S7 Siemens</summary>

#### Description <a href="#description" id="description"></a>

S7 is a communication protocol used by Siemens PLCs (Programmable Logic Controllers). It is used for controlling and monitoring industrial processes.

#### Usage <a href="#usage" id="usage"></a>

* **S7 Client Block**: Use this block to connect to an S7 PLC. Configure the IP address and rack/slot information.
* **S7 Read/Write Blocks**: Use these blocks to read data from or write data to the PLC.

Make sure the PLC is configured correctly and the network settings are appropriate for communication.

</details>

<details>

<summary>MQTT</summary>

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks.

#### Usage <a href="#usage" id="usage"></a>

* **MQTT Subscribe Block**: Use this block to subscribe to a specific topic.
* **MQTT Publish Block**: Use this block to publish messages to a topic.

Ensure the MQTT broker is running and accessible, and the topics are correctly configured.

</details>

<details>

<summary>Modbus</summary>

#### Usage <a href="#usage" id="usage"></a>

* **Modbus Client Block**: Use this block to connect to a Modbus server. Configure the server address and communication settings (e.g., COM port, baud rate).
* **Modbus Read/Write Blocks**: Use these blocks to read from or write to Modbus registers.

Ensure the Modbus server is correctly configured and accessible from AugeLab Studio.

</details>

<details>

<summary>Email</summary>

You can also send e-mail and attachments to multiple people via e-mail block.

* **Send Email** block: Use this block to send e-mail with attachments to multiple people.

</details>


# Block Structures

A function block consists of several elements:

1. Header
2. Sockets
3. Controls
4. Unique Name

<figure><img src="/files/bnqqeLKWNmz72juUkAOa" alt=""><figcaption></figcaption></figure>

## Header <a href="#header" id="header"></a>

Headers can be modified by double-clicking on and setting a custom name.

<figure><img src="/files/WS6vOZhCPkOiqY8TPS6D" alt=""><figcaption></figcaption></figure>

## Tool Tips <a href="#tool-tips" id="tool-tips"></a>

Tool-tips are shown when blocks are hovered on with mouse and display information on inner mechanics of a block.

<figure><img src="/files/lQSy8tUJwHH8jxJdI9dO" alt=""><figcaption></figcaption></figure>

Same rules also apply to Controls. Hovering over [Controls](#Controls) may show additional information:

<figure><img src="/files/Qez305XyD4hrDKzNAHmt" alt=""><figcaption></figcaption></figure>

## Controls <a href="#controls" id="controls"></a>

Controls are custom buttons, lists, or any kind of interactable objects that are shown inside a function block. These widgets can alter the functionality of each function block.

<figure><img src="/files/kGeA9DXVIuLDbsX44wX8" alt=""><figcaption><p>Two same blocks with different widget configurations</p></figcaption></figure>

## Inputs Sockets <a href="#inputs-sockets" id="inputs-sockets"></a>

Data is sent to blocks through input sockets. There are usually multiple sockets in a single block.

Robust data transfer is guaranteed when sockets with same color or category are connected.

<figure><img src="/files/Ac9QHTg1NYFVrrVCZab3" alt=""><figcaption></figcaption></figure>

## Output Sockets <a href="#output-sockets" id="output-sockets"></a>

Results get sent away from outputs and transferred to other blocks by connecting to an input socket.

<figure><img src="/files/5fAnVRzOr0PAsptPYmxL" alt=""><figcaption></figcaption></figure>


# Sockets

## Socket Colors and Types <a href="#socket-colors-and-types" id="socket-colors-and-types"></a>

AugeLab Studio provides different colors in sockets to indicate what kind of data is transferred through a socket. These colors show which class or data type the input/output belongs to. Read the descriptions below to learn which colors are associated with which data types.

{% hint style="info" %}
For more additional information on socket data types, refer to [Coding Reference](/key-features/create-plugins-with-designer-window/coding-reference).
{% endhint %}

### Light Green (Any Image) <a href="#light-green" id="light-green"></a>

<details>

<summary>Light Green (Any Image)</summary>

This socket color corresponds to a mixed image data type and only image data should be connected.

**Camera USB** block below has one green output socket and outputs a colored image it received from the camera.

{% hint style="info" %}
Light Green sockets output BGR and GRAY format data.
{% endhint %}

<figure><img src="/files/erLegKyjLvyDluoko8KY" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Purple (Gray Image)</summary>

Purple sockets correspond to grayscale image type. Color image data cannot be connected to this socket.

{% hint style="info" %}
Purple sockets output grayscale, single-channel image data. This data type may not be used by Light Green sockets. To convert it, use [**Color Space**](/function-blocks/blocks-reference/image-transformations/transformation-filters/color-space) function block.
{% endhint %}

<figure><img src="/files/5fAnVRzOr0PAsptPYmxL" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Blue (Colored Image)</summary>

Blue-colored sockets correspond to colored image data types. These guarantee a colored image output, unlike [Light Green](#light-green) sockets.

{% hint style="info" %}
Colored Image consists of 3 different arrays, Blue-Green-Red. These also can be split with [Split ](/function-blocks/blocks-reference/image-transformations/operations/split-image)[Image](/function-blocks/blocks-reference/image-transformations/operations/split-image) block.
{% endhint %}

<figure><img src="/files/2CEaoiCgGYkAKqeuiaZE" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Light Blue (Boolean)</summary>

Lift Blue colored sockets correspond to logic data types, which are either **True** or **False**.

For example, the camera block above has blue input sockets and only takes True or False values.

{% hint style="info" %}
Logic expressions consist of only two states; **True** and **False**.
{% endhint %}

<figure><img src="/files/AsFdB6CvwTTBb0Uz6fQ3" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Yellow (Number)</summary>

Yellow sockets correspond to integer data type. The following function block has yellow sockets and takes only integer values.

{% hint style="info" %}
Integer basically means whole numbers.
{% endhint %}

<figure><img src="/files/aiqY6oXK1wTVuoQKGri5" alt=""><figcaption></figcaption></figure>

</details>

### Purple (Position) <a href="#purple-position" id="purple-position"></a>

<details>

<summary>Purple (Position)</summary>

Pink sockets output as position/point data type. For example ((x1,y1),(x2,y2)) you can get the position of any object.

{% hint style="info" %}
Point data type consists of two numbers, first is the horizontal position and the second is the vertical position.
{% endhint %}

<figure><img src="/files/LO9pAGp5YmaJeearURbT" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Orange (Shape)</summary>

Orange sockets correspond to the shape data type.

{% hint style="info" %}
Shape data type consists of multiple points, which also consist of two numbers representing where they reside in two-dimensional space.
{% endhint %}

<figure><img src="/files/fxjtXKFhdGpZFt90sqIx" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Dark Green (Undefined)</summary>

Green sockets are sockets corresponding to undefined data types. These color sockets can contain any variant type and can be connected with other sockets.

{% hint style="warning" %}
Be careful when working with green type sockets and make sure data flow is safe.
{% endhint %}

</details>

<details>

<summary>Gray (Text)</summary>

The gray sockets correspond to the text data type.

{% hint style="info" %}
Texts are strings and can be modified to represent results.
{% endhint %}

<figure><img src="/files/kbGQhic3fOMwgVQxWjHm" alt=""><figcaption></figcaption></figure>

</details>


# Blocks Column

Blocks sections includes all the essential functional blocks for you the prepare and deploy your scenarios:

<figure><img src="/files/1sC92vBgoowMHguPuAnV" alt=""><figcaption><p>Blocks Section</p></figcaption></figure>

You may find the detailed description about each block under [All Blocks](/function-blocks/blocks-reference).

Each blocks section is divided into several tabs:

<details>

<summary>Input/Output</summary>

Input/Output section includes blocks that are responsible for supplying data your scenarios and export out.

</details>

<details>

<summary>Data/Logic</summary>

This tab include manipulation of data types outside of images, such as:

* Strings (texts)
* Numbers
* Logical data (true-false)
* Dictionaries, key-value based special containers

</details>

<details>

<summary>Image Transformers</summary>

Here, all function blocks that manipulate image data, such as:

<figure><img src="/files/BWecyeowZ9psPlFPChTT" alt="" width="450"><figcaption><p>Blur Transformer</p></figcaption></figure>

</details>

<details>

<summary>Detections/Shapes</summary>

Here, necessary function blocks for detection of shapes and patterns can be found:

<figure><img src="/files/KnIsb9oWl73iRyz69U62" alt="" width="450"><figcaption><p>Detect Reference Block</p></figcaption></figure>

</details>

<details>

<summary>AI Blocks</summary>

Integrates AI algorithms for advanced image analysis and processing:

<figure><img src="/files/2CELofPjZcAr2Nd8j278" alt="" width="450"><figcaption><p>Detect Reference Block</p></figcaption></figure>

</details>

<details>

<summary>Custom Blocks</summary>

All plugins and user created custom blocks can be shown here.

</details>


# Connections

Connections in AugeLab Studio represent the connections between function blocks:

<figure><img src="/files/onFBxCbSwMkuB4vpFQyE" alt="" width="450"><figcaption></figcaption></figure>

{% hint style="warning" %}
Be cautious of creating recursive connections, as they can lead to potential data loss and workflow disruptions.
{% endhint %}

You should always aim to connect sockets that show similar or same types.

Connections in AugeLab Studio can be manipulation with several shortcuts:

## Moving Existing Connections <a href="#moving-existing-connections" id="moving-existing-connections"></a>

<figure><img src="/files/HTKVLwKCDzBqoZRuB7b2" alt="" width="450"><figcaption></figcaption></figure>

Hover on a socket that is connected, press and hold `Ctrl`, click and hold with `Left Mouse` and move the existing connections to a socket you need, then release.

## Removing Connections <a href="#removing-connections" id="removing-connections"></a>

<figure><img src="/files/wOfftLWlyGltMP9jqLg8" alt="" width="450"><figcaption></figcaption></figure>

You can remove connections by removing them with keyboard shortcuts or clicking `Ctrl+Left Mouse Click` and dragging it.


# All Function Blocks

This section includes references for all blocks.


# AI Blocks


# Face Detection

This function block detects human faces in an input image and returns visual and numeric results for downstream processing.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` The image to analyze for faces (color or grayscale). Provide frames from cameras or loaded images.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` Annotated image with detected face boxes drawn.

`Face Area Coordinates` List of rectangle coordinates for each detected face.

`Face Count` Number of faces detected in the input image.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Confidence Threshold %` A slider to set minimum detection confidence. Increase to reduce false positives, decrease to be more permissive.

## ✨ Features <a href="#features" id="features"></a>

* Real-time face detection suitable for live camera frames or static images.
* Returns both visual feedback (annotated image) and structured data (coordinates and count) for downstream logic.
* Adjustable confidence level to control detection strictness.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect an image-producing block (camera or file loader) to the `Image Any` input.
2. Adjust the `Confidence Threshold %` slider to the desired sensitivity.
3. Use the outputs as needed:
   * Preview the annotated image via a display block.
   * Read `Face Area Coordinates` for ROI processing or tracking.
   * Use `Face Count` for alerts, logging or simple analytics.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

When the block runs, it scans the incoming image for faces above the configured confidence and produces the annotated image, a list of face rectangles, and the detected face count.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For visual inspection, connect this block output to the `Show Image` block to open the image viewer and inspect detections.
* If you only need to monitor one area (e.g., doorway), crop first with `Image ROI Select` to reduce false positives and speed up processing.
* To reduce CPU usage or increase processing speed, insert `Image Resize` before this block to lower frame size.
* Use `Draw Detections` to combine detection rectangles with custom overlays or status text for clearer on-screen results.
* Save examples of successful or failed detections with `Image Logger` for offline review and tuning.
* Preprocess noisy inputs with `Blur` or a thresholding block (like `Image Threshold`) to improve detection stability in low-quality images.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No faces detected: Lower the `Confidence Threshold %` and ensure the image contains clear frontal or slightly angled faces. Try increasing image contrast or use `Image Resize` to a sensible working size.
* Too many false positives: Increase `Confidence Threshold %` and crop the scene with `Image ROI Select` to exclude irrelevant areas.
* Performance issues: Reduce input resolution with `Image Resize` or run detection only on selected frames using a control signal or batching strategy.


# Mask Detection

This function block detects faces and classifies whether masks are worn correctly. It returns an annotated image for visual inspection and numeric counts for quick metrics.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image RGB`\
Image input to analyze. Accepts color images from cameras, files, or previous processing blocks.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image RGB`\
Annotated image with detection overlays for visualization.

`Masked`\
Count of faces detected with correct mask usage.

`Uncorrect Masked`\
Count of faces detected with masks worn incorrectly (e.g., nose uncovered).

`No Mask`\
Count of faces detected without a mask.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Confidence Threshold %`\
Slider to adjust detection confidence. Increase to reduce false positives; decrease to find weaker detections. Use small adjustments to balance precision and recall for your scene.

## 🎯 Features <a href="#features" id="features"></a>

* Visual output with detection overlays for quick review.
* Numeric outputs for easy integration into logging, alarms, or dashboards.
* Runtime confidence tuning via the slider for adaptability to different lighting and distances.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Provide an image to the `Image RGB` input (live or static).
2. Adjust the `Confidence Threshold %` slider to the desired sensitivity.
3. Read the annotated image from the `Image RGB` output and the counts from `Masked`, `Uncorrect Masked`, and `No Mask` for downstream processing.

## ⚙️ Evaluation <a href="#evaluation" id="evaluation"></a>

When the block runs it analyzes the supplied image, annotates detected faces, and outputs the annotated image plus counts of each mask status. Use the confidence slider to tune detection behavior for your environment.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For live camera input use `Camera USB`, `Camera IP (ONVIF)`, or `Stream Reader` as the image source to continuously analyze a stream.
* To inspect results visually while developing, route the annotated image to `Show Image`.
* If you need to process large frames faster, insert `Image Resize` before this block to reduce input size and increase throughput.
* Limit detection to a region of interest using `Image ROI`, `Image ROI Select`, or `Image ROI Polygon` to avoid irrelevant detections and speed up processing.
* Overlay or format detection output for presentations using `Draw Detections` to add rectangles and labels.
* Log or save suspicious frames with `Image Logger`, `Image Write`, or `Record Video` when counts exceed thresholds.
* Combine with `Object_Detection_Tracker` if you need to track mask status over time for the same person (use counts to trigger tracking or logging).
* For batch analysis of stored images, start with `Load Image` and then feed into this block for offline processing.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No detections: try lowering `Confidence Threshold %`, ensure the input image is clear and well lit, or crop to the relevant area with `Image ROI`.
* Many false positives: raise `Confidence Threshold %`, use `Image Resize` to maintain appropriate scale, or restrict analysis with an ROI.
* Slow performance: downscale input with `Image Resize`, or run on fewer frames (skip frames upstream). Use `Image Logger` to capture only frames that meet conditions.
* Unreliable results under poor lighting: improve illumination, or apply preprocessing like `Denoising` and `Contrast Optimization` before feeding images into this block.


# Object Detection - Custom

This function block detects objects in images using custom model files you provide. It lets you load a detector (weights, config, and class list), choose which classes to detect, and control detection sensitivity with a simple slider. The block outputs an annotated image plus structured detection data for further processing.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` The image to be analyzed for object detections.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` Annotated image with detection boxes and labels.

`Object Count` Number of detected objects.

`Object Locations` List of detected object center positions (multiple outputs allowed).

`Object Sizes (w, h)` Width and height for each detected object (multiple outputs allowed).

`Object Class` Class name for each detected object (multiple outputs allowed).

`Rectangles` Bounding rectangle coordinates for each detection (multiple outputs allowed).

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Open Weight File` Button to choose the model weights file.

`Open Config File` Button to choose the model configuration file.

`Open Class File` Button to choose a text file listing class names.

`Class Names` Table where available classes are listed and you can enable/disable each class.

`Confidence Threshold %` Slider to set detection confidence sensitivity (higher = stricter).

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* The block requires three resources to run: a weight file, a config file, and a class list file. Load them using the three file buttons.
* After the files are provided, the detector is initialized and remains ready until you change the files.
* When an image is provided to the input, the block runs the detector on the image and outputs:
  * an annotated image with boxes/labels,
  * count and positions,
  * sizes, classes, and rectangle coordinates for each detection.
* Changing weight or config files triggers reloading of the detector so new models are used for subsequent evaluations.

## ✨ Features <a href="#features" id="features"></a>

* Load-your-own model support (weights, config, and class list).
* Select which classes to detect via an easy checklist.
* Adjustable confidence threshold with immediate effect.
* Outputs both visual results and structured data (counts, positions, sizes, rectangles).
* Works with multiple detected objects and returns results in list form for downstream blocks.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Click `Open Weight File` and select the model weight file.
2. Click `Open Config File` and select the model configuration file.
3. Click `Open Class File` and select the class names file. The class list will populate automatically.
4. Enable only the classes you want to detect in the `Class Names` table.
5. Adjust `Confidence Threshold %` to balance sensitivity vs false positives.
6. Provide the image to the `Image Any` input and run the scenario to get annotated image and detection data.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* To preview results interactively, connect the `Image Any` output to the `Show Image` block.
* If input images are very large and detection is slow, insert `Image Resizer` before this block to lower resolution and increase processing speed.
* Limit analysis to a specific area by using `Image ROI` or `Image ROI Select` upstream so the detector focuses only on regions of interest.
* For tracking detections across frames, link this block’s detection outputs to `Object_Detection_Tracker`.
* If you need custom drawing or overlays beyond the built-in annotations, use `Draw Detections` with the detection rectangles and counts provided by this block.
* Save interesting frames with detections using `Image Logger` or `Image Write` / `Record Video` for later review.
* Monitor performance and GPU usage with `GPU Statistics` when running heavier models.

(hint: enable only required classes and increase the confidence threshold to reduce false positives and speed up post-processing)

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* Missing model files: Ensure all three files (weight, config, class list) are selected. The block cannot run without them.
* No detections: Try lowering `Confidence Threshold %` or enable more classes in the class table. Also verify the class names file matches the model.
* Too many false detections: Increase `Confidence Threshold %` and enable only the relevant classes to reduce noise.
* Slow performance: Reduce input image size with `Image Resizer` or use smaller models; consider offloading to a GPU if available and monitor with `GPU Statistics`.
* Incorrect class names or mismatched files: Verify the class file corresponds to the loaded model (class order and names must match the model training).

## 🔗 Recommended block combinations <a href="#recommended-combinations" id="recommended-combinations"></a>

* `Show Image` — Preview annotation output.
* `Image Resizer` — Speed up detection on large images.
* `Image ROI` / `Image ROI Select` — Focus detection on specific areas.
* `Object_Detection_Tracker` — Track detected objects over time.
* `Draw Detections` — Custom visualization using detection rectangles and counts.
* `Image Logger` / `Image Write` / `Record Video` — Save annotated results for audit or later analysis.


# Object\_Detection\_Tracker

This function block tracks objects across frames using detection results you provide. It assigns consistent IDs to moving objects, maintains a short history for each track (including a stable class label when available), and outputs an annotated image plus lists you can use for counting, logging, or higher-level analytics.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Input Image` The current image frame used for visualization and tracking context.

`Detected Rectangles` A list of detected bounding boxes. Provide the detection rectangles generated by an object detection function block.

`Detected Classes` A list of class names corresponding to the provided detections. Used to build a stable class label per tracked object.

Note: These are input sockets. Feed detections from an object detection block into these sockets.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Result Image` An image annotated with bounding boxes, IDs and class labels for each active track.

`Position - ID List` A list of tracked object center positions with their assigned IDs and stable class names.

`Rectangle - ID List` A list of bounding rectangles paired with assigned IDs and stable class names.

Note: These are output sockets. Use them to visualize, count, or forward tracking data to other blocks.

## 🕹️ Controls <a href="#controls" id="controls"></a>

This function block has no user-facing controls on the block itself. Behavior is driven by the detection inputs you provide and by how upstream blocks configure detection sensitivity and frequency.

## ⚙️ Running mechanism <a href="#how-it-works" id="how-it-works"></a>

* The block receives per-frame detections (rectangles and classes) together with the current frame image.
* It associates new detections with existing tracks to keep identities consistent across frames.
* For each track the block maintains a short history of class detections so that the displayed class becomes stable (it avoids quick class flipping due to momentary misclassifications).
* The block also tolerates short misses (temporary frames without detections) so tracks do not disappear immediately, and it removes tracks that remain inactive for a longer time.
* Outputs include an annotated image for visualization and structured lists (positions and rectangles paired with IDs and class names) for downstream processing.

This behavior is automatic; you control tracking result quality mainly by the quality and frequency of incoming detections.

## 🎯 Key features <a href="#features" id="features"></a>

* Stable ID assignment across frames for each detected object.
* Per-track class stabilization so displayed labels become consistent over time.
* Outputs both visualized results and structured lists for analytics or logging.
* Robust to short detection dropouts (temporary missed detections).
* Creates new tracks for unmatched detections and retires stale tracks automatically.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Connect an image source to the `Input Image` socket (examples: USB camera, IP camera, stream or preloaded frames).
2. Connect a detection block to supply rectangles and class names into `Detected Rectangles` and `Detected Classes` sockets.
3. Use the `Result Image` to preview the tracking output.
4. Use the `Position - ID List` and `Rectangle - ID List` outputs to feed downstream blocks for counting, logging or analytics.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For best results, feed this block with reliable detections from an object detection block such as `Object Detection`, `Object Detection - Custom`, or `Object Detection (D-FINE)`.
* To visualize detections before/after tracking, connect `Result Image` to a `Show Image` or `Draw Detections` block.
* To record visual results, send `Result Image` to `Image Logger` or `Record Video`.
* To focus tracking on a specific area, crop the input first with `Image ROI`, `Image ROI Select` or `Image ROI Polygon` before feeding detections.
* To filter false detections or to count only objects inside a region, combine this block's `Rectangle - ID List` with `Rectangles in Rectangle` or `Check Area (Polygon)`.
* Use the `Rectangle - ID List` output to feed higher-level applications such as `Traffic Intersection Analysis` for traffic counting or area-crossing events.
* If detections are noisy, try adding preprocessing like `Blur`, `Image Threshold` or `HSV Filter` upstream to improve detection quality before tracking.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No tracks appear: Verify that the `Detected Rectangles` and `Detected Classes` sockets receive valid detection data from an object detection block.
* IDs change rapidly or labels flip often: Improve detection stability (better detector, preprocessing) or reduce detection noise so the block can build a stable class history.
* Tracks disappear too quickly: Ensure detections are provided for consecutive frames; if you have intermittent detections, consider smoothing or increasing detection frequency upstream.
* Lots of short-lived tracks: Try filtering small or low-confidence detections before sending them in, or use region cropping (e.g., `Image ROI Select`) to reduce false positives.

If you need a quick example setup: USB camera (`Camera USB`) -> an object detection block (for example `Object Detection (D-FINE)`) -> `Object_Detection_Tracker` -> `Show Image` / `Image Logger` / `Traffic Intersection Analysis`.


# Object Detection

This function block detects common objects in an image and returns both visual and structured detection results. Use it to locate and count items such as people, vehicles, and many COCO classes. It offers quick configuration for confidence and class selection so you can tailor detection to your scenario.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

* `Image Any` This input accepts the image you want to analyze.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

* `Image Any` Image annotated with detection boxes and labels.
* `Object Count` Number of detected objects.
* `Object Center Locations` Locations (centers) of detected objects (can be multiple).
* `Object Sizes (w, h)` Width/height pairs for each detected object (can be multiple).
* `Object Class` Class names for each detected object (can be multiple).
* `Rectangles` Bounding rectangle coordinates for each detection (can be multiple).

## 🕹️ Controls <a href="#controls" id="controls"></a>

* `Confidence Threshold %` Slider to set minimum confidence for accepting detections. Raising this will reduce false positives; lowering it may detect more objects but include less certain results.
* `Select Detection Class` Dropdown to choose a predefined group of classes (for example: All, Human, Animals, Indoor, Outdoor). Selecting a narrower class group speeds up and focuses detection.

## ⚙️ How it runs <a href="#how-it-works" id="how-it-works"></a>

* When the block runs it processes the incoming image with the internal detector and applies the selected confidence and class filter.
* The annotated image is returned together with structured outputs: count, center positions, sizes, class names, and rectangle coordinates.
* If the detector is still loading, the block provides an informative message and waits until the detector is ready before producing results.

## 🎯 Features <a href="#features" id="features"></a>

* Ready-to-use object detection for many common classes.
* Class-group presets to quickly focus on humans, animals, indoor objects, outdoor objects, or all classes.
* Annotated visual output plus detailed numeric/structured outputs for automation or logging.
* Simple controls to balance detection sensitivity and select desired classes.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect a camera or image source to the `Image Any` input.
2. Choose a suitable `Select Detection Class` preset to limit detection to relevant classes.
3. Adjust `Confidence Threshold %` to get the right balance between missing objects and false positives.
4. Use outputs to drive downstream logic, tracking, visualization, or storage.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For live camera input use one of the image input blocks such as Camera sources or stream readers to feed the detector: Camera options include `Camera USB`, `Camera IP (ONVIF)`, `Stream Reader`, or `Load Image` for offline testing.
* To visualize detections in the UI or a dashboard, connect the annotated image output to `Show Image` or draw overlays with `Draw Detections`.
* For tracking objects across frames, feed detection outputs into `Object_Detection_Tracker` to get stable IDs and trajectories.
* If you only care about a specific area, crop the input first with `Image ROI` or `Image ROI Select` to reduce false detections and improve performance.
* When working with very large images, use `Image Resize` or `Image Resizer` before detection to speed up processing.
* Save interesting frames or records by piping the annotated image into `Image Logger`, `Multi Image Write`, or `Record Video` when a detection count or specific class appears. Combine with logic blocks (for example, manual `Logic Input` or threshold checks) to trigger saving only on events.
* Combine with analysis blocks such as `Measure Position Distance` or ROI checks like `Check Area` to build alerts or analytics (for example, count people in a zone or measure spacing).

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If you see too many false positives: increase `Confidence Threshold %` or narrow `Select Detection Class`.
* If detection is slow: resize the input with `Image Resize` or reduce the number of classes searched by selecting a narrower class group.
* If the annotated image looks empty but other outputs show detections: verify the display block (for example `Show Image`) is connected and receiving the annotated image.


# Pose Estimation

This function block detects human body keypoints and (optionally) draws a skeleton on incoming images. Use it to extract selected body part positions for analytics, logging, or visualization.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` The image to analyze (camera frame, loaded image, or preprocessed image).

`Show Skeleton` Boolean input to enable or disable drawing the skeleton on the output image.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` The image with skeleton drawn (if `Show Skeleton` is enabled) and visual markers for detected keypoints.

`Selected Body Part Positions` A dictionary-like result mapping selected body part names to their detected positions (x, y). Only the body part groups you choose are returned.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Class Names` A selectable list of simplified body-part groups (for example: head, chest, elbow, hand, hip, knee, foot). Check the groups you want the block to report.

`Confidence` A slider to adjust detection confidence. Increasing value reduces false positives but may miss faint detections; lowering it makes detection more permissive.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* When an image is provided at `Image Any`, the block analyzes the image and tries to locate human keypoints.
* If `Show Skeleton` is TRUE, the block overlays the skeleton lines and markers on the output image.
* The block returns the annotated image and a mapping of the selected body part names to their detected positions. If a part is not confidently detected it will be omitted from the mapping.

## 🎯 Features <a href="#features" id="features"></a>

* Visual skeleton overlay for quick inspection.
* Selectable body-part groups to limit outputs to only the parts you need.
* Adjustable confidence control to balance sensitivity vs. false detections.
* Real-time friendly for live camera streams when paired with the appropriate image input block.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Provide an image source into `Image Any` (for example: `Camera USB`, `Load Image`, or `Stream Reader`).
2. Choose which body-part groups to output using `Class Names`.
3. Adjust detection sensitivity using the `Confidence` slider.
4. Optionally send a boolean into `Show Skeleton` to enable/disable skeleton drawing.
5. View the result image with `Show Image` or save/log positions for downstream use.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use `Camera USB` as a live image source when working with real-time scenes.
* Resize incoming images with `Image Resize` if the people appear too small — larger person pixels improve keypoint accuracy.
* Run a fast detector first (for example `Object Detection`) and then crop persons with `Image ROI` to feed individual person images to this block — this can increase reliability and lower processing cost.
* To preview results in your workspace, connect this block to `Show Image`.
* Save frames containing detections using `Image Logger` or export coordinates with `Data to JSON` or `CSV Export` for later analysis.
* Combine with `Draw Result On Image` or `Draw Detections` to annotate additional status text or bounding boxes together with skeletons.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No keypoints detected: try lowering the `Confidence` value or increase input image size with `Image Resize`.
* False or jittery keypoints: increase the `Confidence` slider and improve lighting or sharpness (use `Blur` carefully only to remove noise).
* People partially out of frame: use an upstream detection/cropping workflow (for example `Object Detection` → `Image ROI`) so the subject is centered before running this block.
* Slow performance: reduce input resolution with `Image Resize` or process cropped person regions instead of full-frame images.

If you need to log or visualize results, use the suggested combinations under Tips and Tricks to build a robust pipeline.


# Safety Equipment Detection

This function block checks for common safety equipments on an input image and returns an annotated image plus per-class counts. It is intended for visual inspection workflows where helmets, vests, goggles and gloves must be detected and counted.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` Provide the image you want analyzed (single image frames, stream frames, or loaded images).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Output Image` Annotated image with detections drawn for visualization.\
`Helmet Count` Number of detected helmets.\
`Safety Vest Count` Number of detected safety vests.\
`Safety Goggle Count` Number of detected safety goggles.\
`Safety Glove Count` Number of detected safety gloves.\
`No Helmet Count` Number of detected people without helmets.\
`No Safety Vest Count` Number of detected people without vests.\
`No Safety Goggle Count` Number of detected people without goggles.\
`No Safety Glove Count` Number of detected people without gloves.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Confidence Ratio` Adjust detection confidence threshold. Higher values make detections stricter (fewer false positives); lower values increase sensitivity (more detections, possibly more false positives).

Tip: start around 0.7–0.9 and fine-tune based on your scene.

## 🎯 Features <a href="#features" id="features"></a>

* Real-time visual feedback with annotated detections on `Output Image`.
* Per-class counting for both presence and absence of required safety items.
* Adjustable detection confidence using `Confidence Ratio` to suit varying lighting and scene conditions.
* Designed to work with live camera frames or pre-recorded images.

## ⚙️ How it runs <a href="#running-mechanism" id="running-mechanism"></a>

When an image is provided on the `Image` input, the block analyzes the image, marks detected safety items on a visual output image and returns numeric counts for each class. The block may require a short loading time on first run (model preparation), after which it processes images continuously as they arrive.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Provide an image source to the `Image` input (live camera feed or loaded image).
2. Adjust `Confidence Ratio` to suit your scene (lighting, scale, occlusions).
3. Use the annotated `Output Image` to visually confirm detections and read numeric outputs for automation or logging.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Combine with live camera inputs for continuous monitoring: e.g., `Camera USB`, `Camera IP`, or `Stream Reader` to feed frames into this block.
* To visualize and inspect frames interactively, connect the visual output to `Show Image` and use the `See Image` viewer.
* Improve detection focus and reduce false positives by cropping to the area of interest using `Image ROI Select` before feeding images to this block.
* Speed up processing when full resolution is unnecessary by inserting `Image Resizer` before this block.
* Overlay or emphasize detection boxes using `Draw Detections` for clearer on-screen presentation.
* For multi-frame workflows, combine with tracking: feed detection outputs (rectangles / classes) into `Object_Detection_Tracker` to maintain IDs over time and count unique persons.
* Log and export results: use `Image Logger` or `Image Write` to save frames, and `CSV Export` or `Data to JSON` to store counts. For real-time alerts integrate with `MQTT Publish` or `Send Mail` for notifications.
* For crowd or distancing analysis, use together with `Social Distance Detector` or `Pose Estimation` to correlate PPE usage with person location or posture.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No detections or too many false positives: adjust `Confidence Ratio` and re-evaluate. Try values between 0.6 and 0.9.
* Poor results in low-light or low-contrast scenes: improve lighting, use `Contrast Optimization` or `Denoising` prior to this block.
* Detections outside the area of interest: add `Image ROI Select` to limit the search area.
* Slow performance: reduce input image size with `Image Resizer` or lower frame rate upstream. Performance also depends on available hardware.
* If the block is not ready on first use, allow a short time for model preparation (loading) before expecting outputs.


# Social Distance Detector

This function block analyzes an image stream to detect people and check physical distancing based on a distance threshold. It visualizes detected people and highlights pairs that violate the specified distance.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` Feed the image or video frame to be analyzed.

`Perspective Matrix` Optional transformation matrix to convert image coordinates to a real-world plane for more accurate distance measurement.

`Distance Threshold` The minimum allowed distance between people (units depend on perspective calibration).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` Annotated image with detected people, connecting lines, and violation highlights.

`Person Count` Number of people detected in the frame.

`Violation Count` Number of pairwise violations detected (pairs closer than threshold).

`Is Violated ?` Boolean indicating whether any violation exists.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Confidence Ratio` Slider that adjusts detection confidence sensitivity. Higher values require stronger detection confidence to count as a person.

## 🎨 Features <a href="#features" id="features"></a>

* Real-time person detection and visualization on incoming images.
* Pairwise distance measurement between detected people.
* Optional perspective correction using a provided `Perspective Matrix` for more accurate real-world distance checks.
* Clear outputs for monitoring and downstream processing: image, counts, and violation flag.

## 📊 Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

When active, this block receives the image input, detects person positions in the frame, optionally maps those positions using the provided `Perspective Matrix`, computes pairwise distances, and compares them to the provided `Distance Threshold`. The block returns an annotated image and numeric/boolean outputs describing detection and violation state.

## 📝 How to use <a href="#usage" id="usage"></a>

1. Provide an image source to `Image Any` (camera stream or loaded image).
2. If you need real-world distances, supply a calibrated `Perspective Matrix`. Without it, distance checks use image-plane units.
3. Set the desired `Distance Threshold` according to your calibration or approximate pixel distance.
4. Tune `Confidence Ratio` to balance missed detections vs false positives.
5. Read the outputs to trigger alerts, logs, or further processing when violations occur.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For camera input, pair this block with an image source such as `Camera USB`, `Camera IP (ONVIF)`, `Stream Reader`, or `Load Image`.
* To preview results in a larger view, add `Show Image` after this block.
* If detection is noisy, try resizing or denoising the image beforehand using `Image Resizer` or `Denoising` to improve stability.
* For more robust person detection or custom classes, consider combining with `Object Detection` or `Object Detection - Custom` upstream and feed detected centers into this block for distance checking.
* Use `Perspective Transform` to produce a reliable `Perspective Matrix` when you need real-world distances.
* Use `Image ROI` or `Image ROI Select` to limit the analysis area (reduce false detections and speed up processing).
* When tracking is required across frames, use `Object_Detection_Tracker` downstream to get persistent IDs and improved analytics.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No people detected: increase `Confidence Ratio` slightly or provide higher-resolution input. Ensure the scene lighting allows for clear person outlines.
* Many false positives: try increasing `Confidence Ratio`, apply preprocessing like `Blur` or `Image Threshold`, or restrict the area with `Image ROI`.
* Distance measurements seem incorrect: verify the calibration and provide a correct `Perspective Matrix` using `Perspective Transform`. Without perspective correction, distances are in image pixels and may not reflect real-world values.
* High CPU/GPU load: reduce input resolution with `Image Resizer` or run detection less frequently.


# Super Resolution

This function block enhances image quality by upscaling input images using pre-trained super-resolution models. Choose a model that fits your speed and quality needs, then feed an image to get an upscaled result.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` This socket accepts the image you want to enhance.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` The upscaled/enhanced image produced by the block.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`SuperResolution Type` A dropdown to select the upscaling model and scale. Options present different speed vs. quality tradeoffs. Example options include `BEST_x2`, `MEDIUM_x4`, `FAST_x3`, and `FASTEST_x2`. Select the option that fits your hardware and latency requirements.

## 🎨 Features <a href="#features" id="features"></a>

* Upscaling with multiple model choices offering different quality and performance levels.
* Hardware acceleration support for faster processing when a compatible GPU is available.
* Simple one-input / one-output flow for easy insertion into existing pipelines.

## ⚙️ Running mechanism <a href="#how-it-works" id="how-it-works"></a>

* Choose a model from the `SuperResolution Type` control.
* When the block runs, it applies the selected upscaling model to the incoming image and outputs the enlarged image.
* Larger scale factors and higher-quality models will require more processing time and memory.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect an image source to the `Image Any` input.
2. Use the `SuperResolution Type` dropdown to pick a model and scale (for example x2, x3, or x4).
3. Run your scenario to produce the upscaled image on the `Image Any` output.
4. Preview the result with a display or save it with an exporter block.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* If your input images are very large and processing is slow, use `Image Resizer` to reduce the image size before upscaling, or choose a lower scale model such as `FASTEST_x2`.
* To inspect the result visually, connect the output to `Show Image` so you can preview the enhanced image interactively.
* If you want to save processed images for later review, connect the output to `Image Logger` or `Image Write`.
* Apply super resolution only to important regions to save resources: crop with `Image ROI Select` or `Get ROI`, run super resolution on the cropped region, then merge back if needed.
* For downstream tasks that benefit from higher-resolution detail (small object recognition or text reading), try connecting the output to `Object Detection (D-FINE)`, `Object Detection`, `OCR (EasyOCR)`, or `OCR` to improve detection and recognition accuracy.
* If you need to maximize detection throughput, balancing quality and speed is key: prefer `MEDIUM_*` or `FAST_*` models when real-time performance is important.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* GPU out of memory or long processing times
  * Try a lighter option from the `SuperResolution Type` list such as `FASTEST_x2` or reduce the input image size with `Image Resizer`.
  * Close other GPU-intensive applications before running the pipeline.
* Output looks unchanged or artifacts appear
  * Try a different model/scale setting. Higher-quality models improve detail but may introduce different artifacts depending on image content.
  * Consider preprocessing with `Denoising`, `Blur` or `Image Resize` to improve input quality before upscaling.
* Slow evaluation on many images
  * Use `Batch Processing` to control memory use and throughput, or upscale only selected ROIs using `Image ROI Select`.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

On execution, the block produces a single upscaled image reflecting the chosen model and scale. Performance depends on model choice, image size, and available hardware.


# Text Detection

This function block finds text regions in an image and visualizes them. It is tuned for detecting oriented text areas (rotated or tilted) and returns both a preview image and structured location data for further processing.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` Input image to be analyzed for text.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` Annotated image with detected text regions drawn.

`Referance Point` List of reference points (corner points) for each detected text region.

`Referance Rectangles` List of bounding rectangles for each detected text region.

`Number of Detected Text` Total number of text regions found.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Confidence` Adjusts the minimum confidence required for a detection to be accepted. Higher values reduce false positives but may miss faint text.

`NMS Threshold` Adjusts how overlapping detections are merged. Lower values make merging stricter, reducing duplicate boxes over the same text.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

When executed, the block examines the provided image, searches for regions that look like text, filters results by confidence, merges overlapping detections, and scales found regions back to the input image size. The block then outputs an annotated preview image, a list of reference points and rectangles for each detection, and the total count of detected text regions.

## 🎯 Features <a href="#features" id="features"></a>

* Detects rotated and angled text regions, not just horizontal lines.
* Provides both visual feedback (annotated image) and structured outputs (points, rectangles, count) for downstream processing.
* Adjustable sensitivity via `Confidence` and `NMS Threshold` controls to tune precision vs recall.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Provide an image to the `Image Any` input (from a camera, file loader or stream).
2. Adjust the `Confidence` slider to balance false positives vs missed text.
3. Adjust the `NMS Threshold` slider if multiple overlapping boxes appear over the same text area.
4. Use the annotated `Image Any` output to preview detections, and use `Referance Point` / `Referance Rectangles` for extraction, cropping, or passing to recognition blocks.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* If the image is very large, use `Image Resizer` to downscale for faster processing, then map rectangle coordinates back to the original if needed.
* For actual text recognition after detection, connect the detected crop areas to `OCR` or `OCR (EasyOCR)` blocks.
* To focus on a specific area, crop first with `Image ROI` or `Image ROI Select` and feed the cropped image into this block.
* Use `Show Image` to preview the annotated output, and `Draw Result On Image` to overlay custom status text based on detection results.
* Save results with `Image Write` or log examples with `Image Logger` for later review.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No detections: Lower the `Confidence` value or provide a clearer.
* Too many small boxes or duplicates: Increase the `Confidence` value and lower the `NMS Threshold` to merge overlapping detections.
* Detections misplaced after resizing: Ensure any resizing steps are compensated when interpreting `Referance Rectangles` coordinates.
* If you only need recognized text (not locations), chain this block to `OCR` / `OCR (EasyOCR)` and use the count output to gate downstream logic.


# Traffic Intersection Analysis

This function block helps you count and log vehicle movements through defined areas of an intersection. It converts tracked detections and polygonal areas into route statistics (for example, "1-2" means a vehicle moved from area 1 into area 2). Designed for easy integration into an inspection scenario, it outputs summarized tracking data ready for export or display.

## 📥 Inputs (sockets) <a href="#inputs" id="inputs"></a>

* `Rectangle - ID List` — Provide tracked detections (bounding rectangles with tracking IDs).
* `Polygon Areas` — Provide one or more polygon areas that define the intersection zones to monitor.
* `Reset Tracking Log` — Boolean input to clear all stored tracking history and counts.

## 📤 Outputs (sockets) <a href="#outputs" id="outputs"></a>

* `Tracking Data` — Aggregated route counts (JSON-like format) showing how many vehicles of each type passed between area pairs (e.g., "1-2": {"car": 3, "bus": 1}).

## 🕹️ Controls <a href="#controls" id="controls"></a>

* `Reset Tracking Log` — Trigger this input (for example with a manual switch) to reset all counters and internal history.
* (Polygon areas are provided via the `Polygon Areas` socket; use a polygon selection block to define them.)

## 🎯 Key Features <a href="#features" id="features"></a>

* Uses tracked bounding boxes and tracking IDs to associate detections across frames and detect area transitions.
* Keeps a short recent position history per tracked object to stabilize noisy positions before deciding which polygon area an object occupies.
* Produces route-based statistics (area-to-area transitions) grouped by vehicle type.
* Supports multiple polygon areas (each area may have many corner points).
* Reset option to clear history and counters on demand.

## ⚙️ Running mechanism (high level) <a href="#how-it-works" id="how-it-works"></a>

* The block accepts a list of tracked rectangles with IDs and (optionally) vehicle class names.
* For each tracked item it computes a representative point (bottom-center of the bounding box) and maintains a short history of recent points for that ID.
* The representative point (averaged over recent positions) is tested against each supplied polygon area to determine the current area for that vehicle.
* When a vehicle’s area changes (e.g., from area 1 to area 2), the transition is recorded and the corresponding route counter for that vehicle type is incremented.
* The aggregated route counts are output as structured data via the `Tracking Data` socket.
* Activating the `Reset Tracking Log` input clears all stored histories and counters.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Provide tracked detections to the `Rectangle - ID List` input. Use a tracker that outputs rectangles with stable IDs.
2. Define intersection regions with polygons and connect them to the `Polygon Areas` input.
3. To clear counters and history at any time, send a TRUE signal to the `Reset Tracking Log` input.
4. Read the live route statistics from the `Tracking Data` output and use them for display, logging or export.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use `Object_Detection_Tracker` to produce the `Rectangle - ID List` input. This block provides tracked rectangles and IDs that are ideal for route analysis.
* Draw and save polygon areas with `Image ROI Polygon Multi` and connect its `Polygons` output to the block’s `Polygon Areas` input. That block makes it easy to define multiple intersection zones visually.
* Use `Logic Input` to provide a manual trigger for `Reset Tracking Log` when you want to restart counting at known intervals (shift changes, test runs, etc.).
* For visual verification, connect detections to `Draw Detections` to overlay rectangles/IDs on the video, then send the resulting image to `Show Image` to preview results live.
* Export summarized results for reporting with `Data to JSON` or `CSV Export` using the `Tracking Data` output.
* If detections are noisy, consider placing a short smoothing pipeline before tracking (for example, ensure the tracker input image is preprocessed with mild denoising filters) so tracked IDs remain stable.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No transitions recorded: verify that the `Rectangle - ID List` contains stable tracking IDs across frames (unstable IDs prevent detecting area changes).
* Incorrect area assignment: confirm polygon coordinates align with the camera view; use `Image ROI Polygon Multi` to redraw polygons on the actual image.
* Counters not clearing: ensure the `Reset Tracking Log` input is receiving a TRUE signal (e.g., from a `Logic Input` or other control block).
* Missing vehicle type labels: route counts are grouped by vehicle type when that metadata is provided; ensure your tracker outputs class/type along with IDs.


# U2Net Segmentation

This function block performs salient object detection and background removal. Use it to extract a clear foreground mask and to produce a segmented image where the background is removed.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` The input image to analyze (color or grayscale). Connect the image source you want to segment.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Gray` Grayscale probability map / mask showing object likelihood.

`Image Any` Segmented image where background is removed using the mask.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Model` Select between available U2Net variants (for example "U2Net (High Quality)" or "U2NetP (Fast)"). Choose higher quality for better masks or the fast model for speed.

`Input Size` Resize the model input. Smaller values increase speed; larger values improve detail at the cost of performance.

`Threshold` Controls binarization of the probability map to create the final mask. Increasing the threshold makes the mask stricter (fewer foreground pixels).

## 🎨 Features <a href="#features" id="features"></a>

* Foreground extraction to produce a soft probability map and a binary mask.
* Segmented color image output that keeps only the detected foreground.
* Two model choices to trade off accuracy vs speed.
* Adjustable preprocessing size and threshold for fine control.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect an image source to the `Image Any` input.
2. Choose a `Model` depending on whether you prefer quality or speed.
3. Adjust `Input Size` to balance detail and processing time.
4. Tune `Threshold` to get the desired binary mask (preview using a `Show Image` block).
5. Use the `Image Gray` output when you need the mask for further processing, and the `Image Any` output for visualization or saving.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

When run, this block produces a probability mask and a segmented image based on the currently selected model and slider settings. Use the mask output to feed subsequent processing blocks (e.g., filtering, contour detection, ROI operations).

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* To speed up processing while keeping acceptable quality, reduce `Input Size` and use the `U2NetP (Fast)` model.
* For better-looking masks on noisy images, pass the source through `Denoising` or `Blur` before this block.
* If the subject is small relative to the image, use `Image Resize` to upsample the region of interest before segmentation.
* Preview results interactively by connecting the segmented output to `Show Image`.
* Save results automatically by linking the segmented image to `Image Logger` or `Image Write`.
* If you need alternative background removal approaches, compare results with `Background Removal (RMBG-1.4)` or `Background Removal (BiRefNet)` to see which fits your scene best.
* Use the binary mask output as input to ROI or shape-analysis blocks (for example, contour finders or measurement blocks) to extract object geometry.

(hint: useful companion blocks — `Show Image`, `Image Resize`, `Denoising`, `Image Logger`, `Image Write`, `Background Removal (RMBG-1.4)`, `Background Removal (BiRefNet)`)

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If masks look incomplete or too noisy: increase `Input Size` or apply `Denoising`/`Blur` to the input image.
* If the mask is too permissive (background included): raise `Threshold` to make the mask stricter.
* If processing is too slow: switch to the faster model option and/or lower the `Input Size`.
* If no output appears: confirm a valid image is connected to the `Image Any` input and preview using `Show Image`.


# Background Removal (BiRefNet)

This function block performs high-quality foreground / background segmentation using the BiRefNet model. It creates a clean binary mask for foreground regions and can produce a green overlay visualization on top of the original image for quick inspection.

## 📥 Input sockets <a href="#inputs" id="inputs"></a>

`Image` Provide an RGB/BGR image. The block accepts images of any size (larger images may be resized internally for processing).

## 📤 Output sockets <a href="#outputs" id="outputs"></a>

`Overlay` Image visual with segmented foreground shown as a green overlay on the original image.\
`Mask` Binary segmentation mask where foreground pixels are white (255) and background pixels are black (0).

## 🕹️ Controls <a href="#controls" id="controls"></a>

This block runs with automatic settings and does not expose manual widgets inside the block. To influence behavior use helper blocks before or after it, for example:

`Image Resize` — reduce input image size before processing to improve speed and reduce memory use\
`Image ROI` — crop to a region of interest so the model focuses on a smaller area\
`Apply Mask` — combine original image and mask in custom ways after segmentation\
`Show Image` — preview the `Overlay` or `Mask` in a larger viewer\
`Image Logger` / `Image Write` — save masks or overlays for records

## ⚙️ Running mechanism <a href="#how-it-runs" id="how-it-runs"></a>

* The block loads the segmentation model when it is first used. This may download model files and can take time on first run.
* If a GPU is available it will be used to accelerate processing; otherwise CPU will be used.
* For each input image the block returns a binary mask and a colored overlay (green tint on segmented regions).
* Large input images increase processing time and memory usage; resizing the image before feeding it to the block is recommended for faster results.

## 🎯 Features <a href="#features" id="features"></a>

* High-quality segmentation that handles complex boundaries and fine details.
* Produces both a binary mask suitable for further processing and a visual overlay for quick inspection.
* Automatic device selection (GPU if available).
* Works with standard image inputs from cameras, files, or previous processing blocks.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Provide an image to the `Image` input socket.
2. Use the returned `Mask` for downstream processing (measurements, ROI operations, compositing).
3. Use the returned `Overlay` to preview segmentation quality or present results visually.

Suggested flows:

* For faster runs, connect `Image Resize` before this block.
* To segment only a part of the scene, use `Image ROI` first.
* To save results, connect `Image Write` or `Image Logger` to the outputs.
* To preview results during development, attach `Show Image` to the `Overlay` or `Mask`.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* If you want a lightweight alternative, try `Background Removal (RMBG-1.4)` — it uses a different model and may be faster or produce different cutouts depending on the input.
* Clean up small mask artifacts using `Morphological Transformations` or `Image Threshold` after getting the `Mask`.
* If masks look noisy, try running `Denoising` or `Blur` before segmentation to reduce spurious edges.
* To feed high-resolution images but keep memory manageable, use `Image Resizer` to downscale, run segmentation, then use `Image Resize` or `Super Resolution` to scale results back if needed.
* Combine with detection blocks (for example, use results from an object detector to crop the area with `Image ROI` and then run this block for per-object background removal).

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* Model fails to start or is very slow on first use: the block may be downloading model files and initializing. Wait until the download and initialization complete.
* Out of memory errors on GPU: reduce input image size using `Image Resize` or run on CPU by ensuring other GPU workloads are freed.
* Mask has holes or speckles: apply `Morphological Transformations` (fill / open / close) or increase smoothing with `Blur` before segmentation.
* Foreground missing thin structures: try using the alternative `Background Removal (RMBG-1.4)` block to compare results.

Notes:

* Required libraries and models are managed outside of the block; installation of the appropriate packages and access to model downloads are necessary for this block to work.


# Background Removal (RMBG-1.4)

Remove the background from an RGB/BGR image using a pretrained RMBG model. The block returns a foreground cutout and a binary mask that separates foreground (255) from background (0). The model is downloaded automatically on first use.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

* Input socket `Image` RGB/BGR image to process.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

* Output socket `Cutout` Image with background removed (RGB on black or RGBA depending on setting).
* Output socket `Mask` Binary mask image (0 = background, 255 = foreground).

## 🕹️ Controls <a href="#controls" id="controls"></a>

* `Keep RGB on black` Checkbox Controls output format. When unchecked the block provides an RGB cutout on a black background plus the binary mask. When checked the block provides an RGBA cutout with an alpha channel plus the binary mask.

(how the control affects outputs is shown above in the Outputs section)

## 🎨 Features <a href="#features" id="features"></a>

* High-quality background removal using a modern pretrained segmentation model.
* Produces both a visual cutout and a clean binary mask for downstream processing.
* Model is fetched automatically on first run so you can start without manual model handling.
* GPU-aware: will use available GPU resources for faster processing when available.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Provide an RGB/BGR image to the input socket `Image`.
2. Toggle `Keep RGB on black` depending on whether you want an RGB cutout on black background or an RGBA image with alpha channel.
3. Read results from output sockets `Cutout` and `Mask` for visualization or further processing.

## 📊 What the block does when run <a href="#evaluation" id="evaluation"></a>

* Loads the pretrained background-removal model (first run may download required files).
* Processes the input image and outputs a cleaned foreground image and a binary mask.
* If a compatible GPU is available, processing may be accelerated.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use `Show Image` to preview the `Cutout` and `Mask` outputs while tuning settings.
* If you only need a visual overlay, connect the `Mask` output to `Apply Mask` to combine mask with other images.
* Preprocess the input with `Image ROI` or `Image ROI Select` to limit removal to a region of interest — reduces processing time and avoids unintended areas.
* Clean noisy inputs with `Blur` or convert to a more uniform color space with `HSV Filter` before background removal for improved masks.
* If you need a larger result after cutout, run `Image Resize` or `Super Resolution` on the `Cutout` output.
* For an alternative segmentation approach, compare results with `Background Removal (BiRefNet)` and choose the one that gives better edges for your images.
* Save examples and logs by connecting outputs to `Image Logger` or `Image Write` for dataset creation and inspection.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If the block reports missing libraries, ensure required packages are available on the system (the model relies on common ML libraries and image support).
* If results contain holes or missing foreground parts, try preprocessing with `Blur` or crop the image with `Image ROI` to focus the algorithm on the subject.
* If processing is slow, check if a GPU is available and in use; otherwise use smaller images or add `Image Resize` before this block.
* For very fine or complex boundaries, try `Background Removal (BiRefNet)` to compare edge handling.


# Depth Estimation (DepthAny. V2)

This function block estimates per-pixel depth (distance) from a single RGB/BGR image. It produces a colorized depth visualization and a grayscale depth map that you can view, log, or feed to other processing blocks.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` RGB/BGR image used as the input for depth estimation.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Depth Vis` Colorized visualization of estimated depth (easy to inspect).

`Depth Map` Grayscale depth map normalized to 0–255 (useful for analysis and downstream blocks).

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Model Size` Choose the model accuracy / speed trade-off. Options typically include Small, Base, and Large.\
`Max Size` Slider to limit the largest image dimension used during processing. Larger values give more detail but require more processing time.

## 🎨 Features <a href="#features" id="features"></a>

* Produces both a visual depth map and a raw grayscale depth image for flexible use.
* Model selection control lets you trade inference speed for accuracy.
* `Max Size` prevents very large images from slowing down processing; the block automatically resizes inputs for efficient inference.
* Automatically uses available processing resources to give best possible throughput (UI-level behavior).

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

When the block runs it takes the current input image, optionally resizes it to respect the `Max Size` setting, processes it with the selected model size, and returns the two outputs: a color visualization and a normalized grayscale depth map. Larger models and larger `Max Size` values increase processing time.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect an image-producing block to the `Image` input.
2. Select a `Model Size` based on the accuracy vs. speed you need.
3. Adjust `Max Size` to limit processing time while preserving enough detail.
4. Use the `Depth Vis` to quickly inspect results; use `Depth Map` as numeric input for other processing blocks.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For very large images, use `Image Resizer` before this block to reduce processing time while preserving the important area.
* If the input image is low-resolution, try running `Super Resolution` upstream to improve detail before depth estimation.
* To visually inspect outputs in your workflow, connect `Depth Vis` to the `Show Image` block.
* Save results automatically by sending `Depth Vis` or `Depth Map` to `Image Logger` or `Image Write`.
* When experimenting, start with `Model Size` set to Small to get quick feedback, then switch to Base or Large once settings are stable.

(hint: recommended companion blocks — `Image Resizer`, `Super Resolution`, `Show Image`, `Image Logger`, `Image Write`)

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If processing is too slow: lower `Model Size` or reduce `Max Size`.
* If results look coarse: increase `Max Size` or try a larger `Model Size` (if processing resources allow).
* If no output appears or the block stalls: ensure the upstream image block is supplying valid image frames.
* If you need to keep recent results available for review, route outputs to `Image Logger` or `Show Image` to confirm expectations.


# Hand Pose Estimation

This function block detects and estimates hand keypoints (21 per hand) in images and provides both a visual overlay and structured detection data. It is designed for real-time use and offers controls for detection sensitivity, keypoint visibility, style of skeleton output, and the maximum number of hands to process.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` Feed an image (camera frame, loaded image, or processed image) to analyze for hands.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Visualization` Annotated image showing keypoints, skeletons and bounding boxes.

`Hands` Structured detection data (list/dictionary) including bounding boxes, per-keypoint positions, confidence scores and visibility flags.

`Model Info` Basic run-time information such as selected skeleton style and threshold settings.

`Hand Count` Number of hands detected (after applying the configured limits).

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Skeleton Style` Choose how keypoints/skeletons are formatted for the visualization (e.g., MMPose or OpenPose style).

`Det Threshold` Adjust minimum confidence required for a hand detection to be considered valid (0–100 scale).

`Keypoint Threshold` Set the minimum confidence for an individual keypoint to be considered visible (0–100 scale).

`Max Hands` Limit how many hands are returned and drawn (useful to reduce output size and processing for crowded scenes).

## 🎨 Features <a href="#features" id="features"></a>

* Visual overlay with keypoints, skeleton connections and bounding boxes for each detected hand.
* Structured JSON-like output for downstream logic: bounding boxes, per-keypoint (x,y) positions, confidence and visibility.
* User-adjustable thresholds to trade off sensitivity vs. false positives.
* Limit the number of hands processed with `Max Hands` for predictable downstream behavior.
* Automatically uses available hardware to improve speed (will prefer GPU if available).

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Provide an image source into the `Image` input (live camera, stream, or image file).
2. Choose preferred `Skeleton Style` for visualization and downstream format.
3. Adjust `Det Threshold` to control whether weak detections are ignored.
4. Adjust `Keypoint Threshold` to control which keypoints are considered visible.
5. Set `Max Hands` if you only want to track a limited number of hands.
6. Read outputs: use `Visualization` to preview, and use `Hands` / `Hand Count` for logic, UI or logging.

## 📊 How it runs <a href="#evaluation" id="evaluation"></a>

When provided with an image, the block analyzes the picture for hands, applies detection and keypoint confidence thresholds, limits results by `Max Hands`, and then outputs: a visual image with overlays, a structured list of detected hands with bounding boxes and per-keypoint details, a small model info summary, and the number of detected hands.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For live input combine with `Camera USB`, `Camera IP (ONVIF)`, or `Stream Reader` to feed continuous frames.
* Use `Show Image` to preview the `Visualization` output in a larger window while tuning thresholds.
* Preprocess noisy images with `Blur`, `Denoising` or `Image Resize` to improve detection stability.
* If the hands appear cropped or you only want to analyze a specific area, place an `Image ROI Select` or `Image ROI` block before this block.
* To annotate results for reporting, combine `Visualization` with `Write Text On Image` or `Draw Result On Image` and then save with `Image Logger`, `Image Write` or `Record Video`.
* Use `Object Detection` or `Object Detection - Custom` before this block when you want to first locate people and then analyze only person regions for hands—this reduces false positives and speeds processing.
* If you need full-body keypoints as well as hand keypoints, consider pairing with `Skeleton Estimation` and merge results in subsequent processing steps.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No detections: Try lowering `Det Threshold` and `Keypoint Threshold` slightly, or improve image clarity with `Image Resizer`.
* False positives / noisy keypoints: Increase thresholds and/or crop the region of interest with `Image ROI Select` to remove clutter.
* Too slow: Lower image resolution via `Image Resize`, reduce `Max Hands`, or use a faster image source. Using a system with GPU will accelerate processing.
* Missing dependencies or model not available: The block requires the hand-pose model to be available. If the model or runtime components are not present, follow the application’s module installer / module downloader to add the required runtime and model packages.

If you need example combinations or a recommended small pipeline for live hand tracking (camera → preprocess → hand pose → display / save), ask for a suggested block chain and a short explanation.


# Match Anything (ELOFTR)

This function block performs keypoint matching between two images and returns matched keypoints and confidence scores. It can optionally draw matches as a visualization image for quick inspection.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image A` The first image to be matched.

`Image B` The second image to be matched.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Visualization` An annotated image showing matched keypoints and connecting lines (generated only when this output is connected).

`Keypoints A` List of keypoint coordinates detected in `Image A`.

`Keypoints B` List of keypoint coordinates detected in `Image B`.

`Scores` Matching confidence scores for each keypoint pair.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Show Matches` Toggle to draw lines between matched keypoints on the `Visualization` image.

`Threshold` Slider to set matching threshold (0–100). Higher values filter out lower-confidence matches.

## 🎯 Features <a href="#features" id="features"></a>

* Matches keypoints between two images and provides per-match confidence scores.
* Optional visual output that concatenates images and draws keypoints and matching lines.
* Threshold control lets you tune precision vs recall of matches.
* Uses available hardware (CPU or GPU) to run inference when supported.

## 📝 How it runs <a href="#usage" id="usage"></a>

* Provide two images to `Image A` and `Image B`.
* The block computes matching keypoints and scores between the images.
* If the `Visualization` output is connected, an annotated image showing keypoints and (optionally) connecting lines will be produced.
* `Keypoints A`, `Keypoints B`, and `Scores` return lists that can be consumed by downstream blocks.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use `Image Resize` before this block to bring both images to comparable sizes for more stable matching.
* If your source images are low resolution, try preprocessing with `Super Resolution` to improve keypoint quality.
* To inspect the output visually, connect `Visualization` to a `Show Image` block for a full-size viewer.
* Use `Image ROI Select` to focus matching on a region of interest and reduce spurious matches from irrelevant areas.
* Export matched keypoints and scores with `Data to JSON` or `CSV Export` for logging or further analysis.
* Combine with `Image Logger` to save visualizations automatically when matches meet a desired condition.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No matches found: try lowering the `Threshold` to allow more candidate matches, or crop the images with `Image ROI Select` to remove clutter.
* Too many poor matches: raise the `Threshold` to keep only higher-confidence matches.
* Visualization not generated: ensure the `Visualization` output socket is connected — the visual image is created only when requested.
* Missing dependencies or hardware acceleration: this block requires model runtime packages and will run on CPU; if GPU acceleration is expected but not used, check your system GPU availability and the environment where the application runs.


# Object Detection - Custom (CPU)

This function block lets you run a custom object detector on an input image using your own weight, config and class files. Use the provided controls to load model files, choose which classes to detect, and set the confidence threshold. The block returns a visual result image as well as structured detection data (counts, locations, sizes and labels) for downstream use.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image Any` Image to be processed by the custom detector.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Image Any` Annotated image with detections (boxes/labels) drawn.

`Object Count` Number of detected objects in the frame.

`Object Locations` Center positions of detected objects (multiple).

`Object Sizes (w, h)` Width and height of each detected object (multiple).

`Object Class` Detected class names (multiple).

`Rectangles` Bounding rectangle coordinates for each detection (multiple).

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Open Weight File` Button to load the detector weight file (e.g., .weights).

`Open Config File` Button to load the detector configuration file (e.g., .cfg).

`Open Class File` Button to load the class names file (plain text list). After loading, class names populate the class table.

`Class Names` A table showing available classes. Toggle rows to select which classes the detector should report.

`Confidence Threshold %` Slider to set minimum confidence for reported detections. Raise the value to reduce false positives; lower it to capture weaker detections.

## 🎯 Features <a href="#features" id="features"></a>

* Easy loading of custom model resources via UI buttons.
* Selectable subset of classes to focus detection on relevant objects.
* Returns both visual and data outputs: annotated image, counts, positions, sizes, class labels and rectangles.
* CPU-compatible operation for systems without GPU.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Click `Open Weight File` and select your detector weight file.
2. Click `Open Config File` and select the corresponding config file.
3. Click `Open Class File` and load the text file containing class names. The `Class Names` table will populate.
4. Toggle the rows in `Class Names` to choose which classes you want to detect.
5. Feed an image into the `Image Any` input.
6. Adjust `Confidence Threshold %` to control how strict detections must be.
7. Read output values or connect subsequent blocks to react to detections.

## 📊 What the block does when run <a href="#evaluation" id="evaluation"></a>

* Processes the input image with the loaded detector using the selected classes and confidence threshold.
* Produces an annotated image on `Image Any` plus detection data on the other outputs for counting, measurement, tracking, logging or decision-making.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* If you want to visually inspect results in a larger window, connect the `Image Any` output to the `Show Image` block.
* To draw custom overlays or format detection annotations differently, use the `Draw Detections` block with the `Rectangles` and count outputs.
* For higher throughput on large images, insert `Image Resizer` before this block to reduce image resolution—this often speeds up detection with small effect on accuracy.
* Limit the search area to speed up and stabilize results by cropping with `Image ROI Select` or `Image ROI` before feeding the image.
* To keep a record of frames with detections, connect the annotated image output to `Image Logger` or `Image Write`.
* For continuous scenarios where you want to maintain identities across frames, pair this block with `Object_Detection_Tracker` using the detection rectangles and class outputs.
* If you need a quick alternative detector for experimentation, compare results with the `Object Detection` or `Object Detection (D-FINE)` blocks (if available) to find the best trade-off between speed and quality.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No classes listed after loading class file
  * Make sure you selected the correct class file (plain text with one class per line). Use `Open Class File` again if needed.
* Detector reports too many false positives
  * Increase `Confidence Threshold %`. Also try feeding a resized image via `Image Resizer` or restrict the area via `Image ROI Select`.
* Detector misses small or distant objects
  * Lower `Confidence Threshold %` carefully, or pass a higher-resolution image to the block (avoid extremely large sizes—use `Image Resizer` to balance speed vs detail).
* Slow performance on CPU
  * Reduce input resolution with `Image Resizer` or crop the region of interest with `Image ROI Select`. Consider running detection less frequently (e.g., batch processing).
* Need to act on detections in other parts of the flow
  * Use the numeric and structured outputs (`Object Count`, `Object Locations`, `Rectangles`, `Object Class`) to feed logic blocks, logging blocks or tracking blocks like `Object_Detection_Tracker`.


# Object Detection (D-FINE)

This function block performs real-time object detection on an input image. It lets you choose a model size (trade-off between speed and accuracy), filter which object classes to detect, set a confidence threshold, and optionally draw color-coded bounding boxes on the output image.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` The image to analyze for object detections.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Result` Annotated image with bounding boxes and labels (present if drawing is enabled).

`Boxes` A list of bounding box coordinates for each detection (format: \[x1, y1, x2, y2]).

`Labels` Class names for each detected object.

`Scores` Confidence score for each detection.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Model` Choose model size (examples: Nano / Small / Medium / Large / XLarge) to balance inference speed vs accuracy.

`Select Classes` Table to tick only the COCO classes you want the block to report (leave all unchecked to allow all classes).

`Draw Boxes` Toggle to enable/disable drawing colored bounding boxes and labels on the output image.

`Threshold` Slider to set the confidence score threshold (0–100) for accepting detections.

## 🎨 Features <a href="#features" id="features"></a>

* Model size selection to prioritize speed or accuracy.
* Class filtering so you only get detections you care about.
* Confidence threshold control to reduce false positives.
* Optional visual output with color-coded bounding boxes and readable labels.
* Returns both visual (image) and structured detection data (boxes, labels, scores) for downstream processing.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Connect an image-producing block to the `Image` input.
2. Choose a `Model` size based on whether you need faster results or higher accuracy.
3. Use `Select Classes` to limit detection to only the object types you need (or leave empty to accept all).
4. Adjust the `Threshold` slider to tune detection sensitivity.
5. Enable `Draw Boxes` if you want a visual result returned on the `Result` output.
6. Use the `Boxes`, `Labels`, and `Scores` outputs in subsequent processing or logging blocks.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

When run, this block processes the incoming image according to the chosen `Model` and `Threshold` and outputs detections. If `Draw Boxes` is enabled, an annotated image will be available on `Result`; detection metadata is always available on the other outputs.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* If small objects are missed, try feeding a higher-detail input using `Super Resolution` or avoid downscaling the input with `Image Resize`.
* To reduce processing time, pick a smaller `Model` (Nano/Small) at the cost of some accuracy.
* Crop the area of interest before detection with `Image ROI Select` or `Image ROI` to improve speed and reduce false positives.
* For visual debugging, connect the `Result` output to `Show Image` so you can inspect detections interactively.
* If you need to keep detections across frames (tracking), combine outputs with `Object_Detection_Tracker`.
* Save frames with detections using `Image Logger`, `Image Write`, or `Record Video` for later review.
* Export detection metadata (boxes / labels / scores) with `CSV Export` or `Data to JSON` for analytics and reporting.
* When experimenting, monitor system usage with `GPU Statistics` to choose an appropriate `Model` size for your hardware.
* If you want different detection behavior or to try other models, compare outputs with `Object Detection - Custom` or other object detection alternatives.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If you see no detections, lower the `Threshold` or make sure the desired class is checked in `Select Classes`.
* If detections are noisy, raise the `Threshold` or limit classes in `Select Classes`.
* If processing is too slow, choose a smaller `Model` or crop the input image using `Image ROI Select`.
* If the output image is empty while other outputs contain data, ensure `Draw Boxes` is enabled to get a visual result.


# OCR (EasyOCR)

A function block that detects text in images using the EasyOCR engine. It can return recognized text, individual text segments, bounding boxes, and an optional annotated image. Includes automatic rotation attempts to handle rotated text.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` Grayscale or color image containing text to be recognized.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Result` Annotated image with detected text boxes and optional text labels.

`Whole Text` All recognized text joined as a single string.

`Texts` List of recognized text segments.

`Boxes` List of polygon boxes for each detected text (4 points per box).

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Use auto rotation` Enable attempts to rotate the image (90°, -90°, 180°) to improve detection on rotated text.

`Show Texts` Toggle display of recognized text labels on the annotated `Result` image.

`Threshold` Confidence threshold slider to filter out low-confidence text detections (0–100%).

## 🎯 Features <a href="#features" id="features"></a>

* Multi-angle detection: tries simple rotations to handle rotated text when `Use auto rotation` is enabled.
* Confidence filtering: remove weak detections by adjusting `Threshold`.
* Visual output: returns an annotated `Result` image with boxes and optional text labels when detections are produced.
* Returns both aggregated `Whole Text` and individual `Texts` and `Boxes` for downstream processing.

## 📝 Usage <a href="#usage" id="usage"></a>

1. Connect an image-producing block into the `Image` socket.
2. Optionally enable `Use auto rotation` if source images may be rotated.
3. Adjust `Threshold` to balance sensitivity vs false detections.
4. Enable `Show Texts` if you want text labels drawn on the `Result` image.
5. Use the outputs to save, log, or further process recognized text and box coordinates.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

When executed, the block analyzes the input image (and rotated variants if enabled), filters detections by the `Threshold`, and returns the annotated image (if any), the full concatenated text, a list of text segments, and the list of polygon boxes.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Preprocess low-quality images with `Image Resizer` or `Super Resolution` to improve recognition on small or blurred text.
* Crop to the text area first using `Image ROI Select` or `Image ROI` so the block focuses on relevant regions.
* Reduce background clutter with `Background Removal (RMBG-1.4)` or `Background Removal (BiRefNet)` before OCR when text sits on complex backgrounds.
* Improve contrast and readability using `Auto Contrast`, `Adjust Colors`, `Denoising`, or `Image Threshold` depending on the input.
* For live inspection, route the `Result` to `Show Image` to preview annotated detections.
* Save detection images or logs with `Image Logger`, `Image Write`, or export recognized text using `Data to JSON` or `CSV Export`.
* If you need alternative OCR behavior, compare results with the generic `OCR` block or combine with `Draw Detections` to overlay custom visuals.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No text detected: lower the `Threshold` and ensure the input image is not too small — try `Image Resizer` or `Super Resolution`.
* False positives or garbled text: raise the `Threshold` and apply `Denoising` or `Auto Contrast` to stabilize inputs.
* Rotated text not found: enable `Use auto rotation` to try common rotations, or pre-align using `Auto Alignment` if you have a reference.
* Too many small detections: crop to the region of interest with `Image ROI Select` and filter results by bounding box size in downstream blocks.


# OCR

This function block extracts text from images and returns recognized text, bounding boxes, and an optional annotated image. It is designed for simple drag-and-drop pipelines: feed an image, set confidence threshold and options, then consume the text or visualize results.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` This input accepts any image you want to perform text recognition on.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Result` Annotated image with detection boxes and optional text overlay.

`Whole Text` All detected text joined as a single string.

`Texts` Individual detected text strings (multiple outputs possible).

`Boxes` Detected bounding boxes for each text region.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Use auto rotation` Toggle to enable automatic rotation handling for upside-down / rotated text.

`Show Texts` Toggle to draw recognized text labels on the annotated output image.

`Threshold` Slider to set minimum confidence for accepted detections (higher = fewer, more confident results).

## 🎨 Features <a href="#features" id="features"></a>

* Real-time text detection and recognition from images provided to the `Image` input.
* Confidence filtering using the `Threshold` slider to ignore weak detections.
* Optional image annotation: draw boxes and text on the returned `Result` image when `Show Texts` is enabled.
* Auto-rotation support to improve recognition on rotated or upside-down text when `Use auto rotation` is enabled.
* Outputs are provided in both human-readable string form (`Whole Text` / `Texts`) and structured geometry form (`Boxes`) for downstream processing.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* Provide an image to the `Image` input and run the scenario.
* The block processes the image and filters detections below the chosen `Threshold`.
* If any outputs that require visualization are connected, the block returns an annotated `Result` image with boxes and optional text overlays.
* Text outputs (`Whole Text`, `Texts`) and geometry outputs (`Boxes`) are produced for use by other blocks or export tools.
* Note: the first run may take longer (model or resources are initialized on demand). Afterwards, inference runs faster.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Feed the image you want to read into the `Image` input.
2. Set the `Threshold` slider to filter out low-confidence recognitions. Start with moderate values (e.g., 30–50) and adjust.
3. Enable `Use auto rotation` if your images may contain rotated text.
4. Enable `Show Texts` if you want the returned `Result` image to show detected text labels.
5. Use the `Whole Text`, `Texts` or `Boxes` outputs as needed (for display, logging, or logic).

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Improve accuracy by cropping the area that contains text first with `Image ROI Select` so the block only processes the region of interest.
* Speed up inference on large images by using `Image Resize` before feeding the image into this block.
* Remove complex backgrounds or isolate text regions with `Background Removal (RMBG-1.4)` or `Background Removal (BiRefNet)` when text sits on cluttered backgrounds.
* Reduce visual noise with `Blur` or convert to a cleaner binary image with `Image Threshold` when working with noisy captures.
* Quickly preview OCR results by sending the annotated `Result` image to `Show Image`.
* If you want an alternative OCR approach, try the `OCR (EasyOCR)` block and compare results for your dataset.
* Save recognized text or images downstream with `Image Logger`, `Image Write`, or `CSV Export` for traceability.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No text detected: increase the input image quality (crop to text, increase resolution) or lower the `Threshold` slightly.
* Many false detections: raise the `Threshold` and crop the image to focus on real text regions.
* Rotated text not read correctly: enable `Use auto rotation` or pre-rotate the image with `Image AutoRotator` or an `Image Resize` + rotate step.
* Slow initial run: the first evaluation may take longer while resources initialize. Subsequent runs will be faster.
* If you only need bounding geometry for logic (not annotated images), disconnect visualization outputs to reduce processing overhead.


# Skeleton Estimation

This function block performs full-body skeleton estimation on input images. It provides multiple detail levels (Body, Body with Feet, Wholebody), adjustable performance modes, and confidence thresholds so you can balance speed and accuracy. Results include a visualization image, structured skeleton data, model metadata, and a person count.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Image` The image to analyze for human poses. Accepts typical image sources (camera frames, loaded images, or processed images from other blocks).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Visualization` An image with skeletons and bounding boxes drawn for detected people.

`Skeletons` Structured data describing detected persons, their bounding boxes, keypoints (with names and confidences), and optional body-part groupings.

`Model Info` Metadata about the current model selection and runtime settings (model type, mode, device, thresholds).

`Person Count` Number of detected persons included in the structured output.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Model Type` Choose the detail level: `Body` (17 keypoints), `Body with Feet` (26 keypoints), or `Wholebody` (body + face + hands).

`Mode` Select processing profile for speed vs accuracy, e.g. `lightweight`, `balanced`, `performance`.

`Skeleton Style` Choose keypoint output style, for example `MMPose` or `OpenPose` formats.

`Detection Threshold` Adjust minimum confidence required to consider a detected person valid.

`Keypoint Threshold` Adjust minimum confidence required to mark individual keypoints as visible.

`Max Persons` Limit the number of people processed and returned to keep performance stable.

## 🎯 Key Features <a href="#features" id="features"></a>

* Multiple model formats to suit your use case: quick body-only detection or detailed whole-body analysis (face + hands).
* Performance tuning via `Mode`, `Detection Threshold`, and `Max Persons` to adapt to device capabilities.
* Confidence-based keypoint filtering so only reliable keypoints are reported as visible.
* Visual feedback with skeleton overlays and bounding boxes for easy verification.
* Structured outputs suitable for downstream automation, analytics, or logging.

## ⚙️ Running Mechanism (User-Facing) <a href="#how-it-works" id="how-it-works"></a>

* When an image is provided, the block runs the selected estimation model and returns both an annotated image and structured pose data.
* `Detection Threshold` controls whether a detected person is considered valid. Lower values return more detections but may include false positives; higher values are stricter.
* `Keypoint Threshold` controls which keypoints are flagged as visible; use this to ignore low-confidence joints.
* `Max Persons` truncates results to the top detections to preserve performance on crowded scenes.
* The block adapts to the chosen `Mode` to trade off speed and accuracy: choose lighter modes for real-time needs and heavier modes for accuracy.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Provide an image source to `Image`. Typical sources are camera frames (e.g. `Camera USB` or `Camera IP (ONVIF)`) or a loaded image (`Load Image`).
2. Choose the desired `Model Type` depending on the detail you need.
3. Set `Mode` to match your performance expectations (faster or more accurate).
4. Tune `Detection Threshold` and `Keypoint Threshold` to filter unreliable detections.
5. Optionally reduce `Max Persons` for faster processing on resource-limited systems.
6. Use the outputs: visualize `Visualization`, send `Skeletons` to analytics or logging, and monitor `Person Count`.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* For live setups, use `Camera USB`, `Camera IP (ONVIF)`, or `Stream Reader` as image sources. For testing, use `Load Image`.
* If processing is slow, try:
  * Selecting a faster `Mode`.
  * Reducing `Max Persons`.
  * Pre-resizing images with `Image Resize` before feeding into this block.
* Improve robustness in noisy images by applying preprocessing such as `Blur` or `Denoising` before the skeleton block.
* To focus on a specific area (e.g., a doorway or assembly line), crop with `Image ROI` or `Image ROI Select` and run skeleton estimation only on that region.
* Combine outputs with visualization and logging blocks:
  * Send `Visualization` to `Show Image` for an interactive preview.
  * Overlay bounding boxes or labels using `Draw Detections` or `Write Text On Image` to create clear operator displays.
  * Save verification frames with `Image Logger` or record sessions with `Record Video` for audits.
  * Convert structured `Skeletons` data into logs using `Data to JSON` or export counts via `CSV Export`.
* For higher-level safety or analytics:
  * Use `Skeletons` (person positions) together with `Social Distance Detector` to check proximity violations (you may need a perspective transform or calibration via `Perspective Transform`).
  * Feed person bounding boxes or centers into custom logic blocks to trigger alerts or external actions (e.g., `Send Mail` or `MQTT Publish`).

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If you get few or no detections:
  * Increase `Detection Threshold` and/or `Keypoint Threshold` incrementally, or try a more accurate `Mode`.
  * Ensure the subject is well-lit and clearly visible in the `Image` source.
  * Try preprocessing with `Image Resize` (upsample or downsample) to match the expected subject scale.
* If performance or responsiveness is poor:
  * Pick a lighter `Mode`, lower `Max Persons`, or preprocess images to smaller sizes with `Image Resize`.
* If results look noisy or jittery across frames:
  * Consider adding temporal smoothing in downstream logic or only logging confident detections (use thresholds).
* If model initialization or runtime fails: ensure required runtime components are available (installable via the application’s module tools) and then re-run the block.

## 🔗 Example Block Flows <a href="#example-combinations" id="example-combinations"></a>

* Real-time monitoring: `Camera USB` → `Image Resize` → `Skeleton Estimation` → `Draw Detections` → `Show Image`
* Logged audit trail: `Camera IP (ONVIF)` → `Skeleton Estimation` → `Image Logger` + `Data to JSON`
* Safety enforcement (distance checks): `Camera USB` → `Skeleton Estimation` → (extract person centers) → `Social Distance Detector` → `Draw Result On Image`

Use these combinations to build reliable and performant pose-detection systems without needing to touch implementation details.


# Data/Logic


# Flow Control


# Batch Concatenation

This function block merges multiple batch/list inputs into a single combined batch. Use it when you want to join several lists of items (for example images, detections, or generic data lists) into one list for downstream processing.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

* `Batch`\
  One or more batch/list inputs. Each input can contain a list of items (images, detections, generic values). Inputs are accepted as batch sockets.
* `Batch`\
  A second batch/list input (the block accepts multiple batch inputs when added).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

* `Batch`\
  A single batch/list output that contains the concatenation of all provided input lists, preserving item order.

## 🕹️ Controls <a href="#controls" id="controls"></a>

This function block has no user-facing controls. It operates automatically using the provided inputs.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

When the scenario runs, the block collects the lists provided on its batch inputs, merges them by appending items from each input in order, and outputs one combined batch. Empty inputs result in an unchanged merged list (items from other inputs are preserved).

## 🎯 Features <a href="#features" id="features"></a>

* Simple list merging: concatenates multiple lists into a single list.
* Preserves original order of items from each input.
* Accepts generic batch/list items (images, numbers, dictionaries, etc.).

## 📝 How to use <a href="#usage" id="usage"></a>

1. Feed one or more batch/list outputs into the block's batch inputs.
2. Connect the block's single batch output to downstream blocks that expect a list or batch.
3. Use this when you need to combine parallel branches (for example, outputs from separate cameras or parallel detectors) back into a single processing stream.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Combine with `Debatch` when you need to split a merged batch back into individual elements for per-item processing.
* Use `Get Batch Size` to inspect how many items are in the merged batch.
* Use `Get Element` to extract specific items from the merged batch by index.
* If you need to process large lists memory-efficiently, pair with `Batch Processing` to handle items in smaller chunks.
* To filter out missing or invalid entries before merging, use `Exclude Nones` or `Replace None` on incoming lists.
* Common downstream consumers of merged image batches include `Collage Images`, `Image Concatenate`, `Image Logger`, or any detector/analysis block that accepts image lists.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If the merged batch appears empty: verify each input batch contains items and that inputs are actually connected.
* If order matters, ensure inputs are provided in the desired sequence since the block preserves input order when concatenating.
* If some items are unexpected (e.g., None), use filtering blocks like `Exclude Nones` before merging.


# Batch Processing

This function block collects multiple input values into a single batch container to reduce memory usage during processing. Use it when you want to group items (images, data, or generic values) into a single stream that other blocks can consume as a batch.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Input 1`\
First value to be included in the batch. Can be image data, numbers, text, lists, or other generic values.

`Input 2`\
Second value to be included in the batch. Additional inputs can be connected depending on your workflow.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Batch`\
A single batched list that contains the connected input values grouped for batch processing.

## 🕹️ Controls <a href="#controls" id="controls"></a>

This function block has no interactive controls. It works by grouping whatever is connected to its input sockets.

## 🎯 Features <a href="#features" id="features"></a>

* Groups multiple inputs into a single, memory-friendly batch object for downstream processing.
* Accepts generic data types so it can batch images, numbers, text or lists.
* Useful for lowering memory footprint when handling many items in a pipeline.

## 📝 How to use <a href="#usage" id="usage"></a>

1. Connect the items you want to group into `Input 1` and `Input 2` (or more inputs if available).
2. The block will output a single `Batch` that contains those inputs as a list.
3. Feed the `Batch` output into blocks that accept batch-style input or into blocks that can iterate over the batch.

## ⚙️ Running behavior <a href="#evaluation" id="evaluation"></a>

When the scenario runs, the function block collects values present at its input sockets and emits them bundled as a single `Batch`. If an input is missing or invalid, the batch will include a placeholder for that entry so downstream blocks can handle it consistently.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Create grouped image sets from multiple sources by combining image inputs like `Load Image`, `Camera USB`, `Camera IP (ONVIF)`, or `Video` into a single `Batch` for processing.
* Use `Batch Processing` before heavy AI blocks such as `Object Detection`, `Mask Detection`, `Super Resolution`, or `OCR` to reduce peak memory usage when running many images.
* After processing a batch, use `Debatch` to split the results back into individual items for drawing or saving.
* Merge multiple batches with `Batch Concatenation` when you need to combine batches created in different parts of a workflow.
* Use `Get Batch Size` and `Get Element` to inspect or access items inside the `Batch` for conditional logic or selective processing.
* When saving results, connect the per-item outputs (after `Debatch`) to `Image Logger`, `Image Write`, or `Record Video` to store processed images efficiently.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No data in `Batch`: Verify the upstream blocks are producing values on the inputs. If an input is intentionally empty, that empty slot will appear in the batch.
* Unexpected item order: The batch preserves the order of inputs as connected. Reorder connections if a different sequence is required.
* Downstream blocks not accepting the batch: Some blocks expect single items rather than batches. Use `Debatch` to convert a batch back to individual items before feeding those blocks.


# Debatch

This function block disables batch processing for the current variable and demultiplexes (demuxes) a list/batch input into separate outputs so downstream blocks receive individual elements.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Batch` This input socket accepts a list or batch of values (for example: a list of images, numbers, shapes or generic items). The block expects the batch to be provided here.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Output 1`\
`Output 2`

These output sockets provide the individual elements extracted from the incoming batch in sequential order. If the batch contains fewer elements than available outputs, remaining outputs will be left at default/empty values. If the batch contains more elements than available outputs, the first elements are provided to the outputs in order.

## 🕹️ Controls <a href="#controls" id="controls"></a>

This block has no interactive controls or user widgets.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* When a valid batch (list) is connected to the `Batch` input, the block splits the batch and emits elements through the output sockets in order.
* If the input is missing or invalid, the block will show an error/invalid state in the UI and provide default outputs to indicate no valid data is available.
* Use this block when you need to turn a grouped list of items into individual streams so other blocks can process each item independently.

## 🎯 Features <a href="#features" id="features"></a>

* Simple demultiplexing of batch/list data into separate outputs.
* Works with any generic data type (images, numbers, shapes, etc.).
* Safe fallback behavior when input is not connected or invalid.

## 📝 Usage instructions <a href="#usage" id="usage"></a>

1. Provide a batch or list of items to the `Batch` input.
2. Connect downstream blocks to `Output 1` and/or `Output 2` to receive individual elements.
3. If you expect varying batch sizes, add logic or checking blocks downstream to handle missing or extra elements gracefully.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Combine with `Batch Processing` to switch between batched and per-item processing: produce a batch with `Batch Processing` then use this block to process items individually.
* Use `Get Batch Size` or `Get Element` before or after this block to inspect batch length or access specific elements.
* Reassemble processed items using `Batch Concatenation` or `Mux` when you need to create a single batch/list again.
* Pair with image-focused blocks: feed a batch of images into this block, then connect each output to blocks such as `Show Image`, `Image Logger`, `Find Object`, `Object Detection`, or `Mask Detection` for per-image analysis and saving.
* When working with optional values, chain `Exclude Nones` after this block to remove empty elements before further processing.
* If you need to limit or transform the element stream, use it together with `Get Element` and logic blocks (for example: filtering, counters) to control which elements are passed on.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No outputs update: verify that the `Batch` input is connected and contains a list of items.
* Unexpected empty outputs: check the batch size; if it has fewer elements than the number of outputs, some outputs will be empty by design. Use `Get Batch Size` to confirm the incoming batch length.
* Downstream errors: ensure downstream blocks accept the specific element type (image, number, shape, etc.) emitted by this block. Use type-conversion or validation blocks if needed.


# Get Batch Size

This function block returns the number of elements in a provided batch or list. It is useful for flow control, logging, and branching decisions when working with grouped data.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

* `Batch` This input accepts a batch/list of items to measure.
  * Socket: Input

## 📤 Outputs <a href="#outputs" id="outputs"></a>

* `Size` Numeric value representing the count of elements in the provided batch.
  * Socket: Output

## 🕹️ Controls <a href="#controls" id="controls"></a>

* `No Controls` This function block does not expose any interactive widgets.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

When this block receives a batch on its input socket it evaluates the collection and outputs the current number of elements on the `Size` output. Evaluate this block downstream of any process that creates or combines batches to get an immediate numeric count for decision making or logging.

Use helper blocks (see Tips and Tricks) to validate or adjust the batch before feeding it in, so the size measurement is reliable even when upstream data is missing or dynamic.

## 🎯 Features <a href="#features" id="features"></a>

* Simple, single-purpose block that provides an immediate numeric size for any batch/list input.
* Works as a lightweight flow-control tool to trigger conditional logic based on batch length.
* Compatible with generic batch-producing blocks in the flow control and data processing categories.

## 📝 Usage <a href="#usage" id="usage"></a>

* Place this block after any block that outputs a list/batch to obtain the number of items being passed along.
* Feed the `Size` output into comparison or logic blocks (for example, `Greater`, `Equals`, `Logic Input`) to branch the scenario based on batch count.
* Combine with logging or export blocks to record how many items were processed in each run.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use with `Batch Processing` when you are creating batches for memory-efficient processing; connect its batch output to `Batch` here to monitor batch sizes.
* After splitting a batch with `Debatch`, reconnect pieces and use this block to verify the size of reassembled lists.
* Combine with `Get Element` to check a batch length before trying to access an index; this prevents out-of-range access.
* When merging multiple batches, use `Batch Concatenation` first, then this block to confirm the resulting size.
* If the upstream source may provide no data, pair this block with `Is None` or `Replace None` to ensure stable behavior and avoid unexpected errors.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If the block reports unexpected sizes, verify that the input is indeed a list/batch type (use debugging or `Debug Input` to inspect content).
* If the input may be absent, add validation using `Is None` or `Replace None` so the size measurement remains predictable.


# HMI Background

This function block provides a large, resizable visual canvas intended for building Human-Machine Interface (HMI) layouts inside your scenario. It acts as a background graphic where you can place and organize interactive blocks and visual elements to create a dashboard-like interface.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

This block does not have any inputs.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

This block does not have any outputs.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Resize` Adjust the block size in the editor to fit your desired HMI layout area.

`Background Canvas` The visual area where you arrange and preview other HMI elements and visual blocks.

## 🎨 Features <a href="#features" id="features"></a>

* Visual-only background for laying out HMI screens and dashboards.
* Resizable canvas so you can design compact or expansive interfaces.
* Designed to host and visually organize other blocks and display elements (images, text, indicators) for operator panels and demos.
* Lightweight: it does not process image or sensor data itself; it exists to structure and present other blocks.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* This block does not produce data or perform computations during scenario runs.
* Its purpose is purely graphical and organizational: use it to visually group and position interactive and output blocks that do produce data when the scenario runs.
* Because it is visual-only, placing heavy processing elements inside the same visual area does not change the background’s behavior—processing still happens in the connected function blocks.

## 📝 Usage <a href="#usage" id="usage"></a>

1. Add the `HMI Background` block as a canvas for your interface.
2. Resize the block to match the screen area you want to design.
3. Place display and control blocks over the background to build your HMI layout (for example, image viewers, text overlays, buttons, and indicators).
4. Use structural blocks to keep logic behind the interface tidy and reusable.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Group logic and presentation: Use `Subsystem`, `Subsystem In`, and `Subsystem Out` to keep the processing logic separated from the visual layout placed on the background.
* Visualize camera feeds: Combine with `Show Image` to display live camera frames on the HMI canvas.
* Add status indicators and labels: Use `Write Text On Image` and `Led Output` to show dynamic text and boolean status indicators on top of displayed images.
* Provide user controls: Place `Logic Input`, `Number Input`, or `String Input` blocks near the related displays so operators can interact with the scenario.
* Save evidence and logs: Use `Image Logger` or `Record Video` (when available) together with displayed images to archive important frames triggered from your HMI.
* Keep UI responsive: Place only visualization and input blocks on the background. Heavy processing (detection, segmentation, tracking) should remain in separate functional blocks.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* Background not visible or too small: Try increasing the block size with the `Resize` control.
* Layout feels cluttered: Use `Subsystem` blocks to collapse and hide complex logic while keeping the HMI canvas clean.
* Slow editor interaction: Remove or relocate heavy processing blocks out of the immediate visual area; keep the background focused on presentation and control elements.


# Subsystem Enabled

This function block provides an enclosed subsystem that can be run conditionally. Use it to group a collection of blocks (a small scenario) and control when that collection executes. It is ideal for building optional processing branches, feature toggles, or guarded workflows that should run only when explicitly enabled.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Enable`\
A boolean input that controls whether the enclosed subsystem runs.

* If this input is connected and set to False, the enclosed subsystem will be skipped.
* If this input is not connected or set to True, the enclosed subsystem will run.

(The block accepts further inputs indirectly by exposing inputs of the enclosed blocks.)

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Dynamic Outputs`\
This block does not declare fixed outputs. Instead, when the enclosed subsystem runs it exposes the outputs produced by the blocks inside it. Think of this block as a container: its outputs mirror whatever outputs the inner blocks provide.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Subsystem Canvas`\
A visual area where you can open or edit the enclosed subsystem. Drop blocks inside this canvas to build the internal flow.

`Enable`\
The input socket described above acts as the primary control for turning the subsystem on or off.

(There are no extra sliders or buttons on the block itself; all controls are the blocks you place inside the subsystem and the `Enable` input you provide.)

## 🎨 Features <a href="#features" id="features"></a>

* Encapsulation — group related processing steps into a single reusable unit that can be enabled or disabled.
* Conditional execution — run the internal workflow only when desired, simplifying top-level graphs.
* Dynamic outputs — the block forwards outputs from its internal blocks so the top-level flow can consume them when the subsystem is active.
* Visual separation — keeps the main flow tidy by hiding detailed processing inside the subsystem canvas.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* If the `Enable` input is connected and False, the subsystem is not executed and the block will provide its default outputs.
* If the `Enable` input is not connected or is True, the subsystem is executed and the outputs from internal blocks are collected and exposed.
* The block monitors internal blocks for invalid states; if any internal block reports an error the block will indicate an invalid state so you can inspect and fix the subsystem.

## 📝 How to use <a href="#usage" id="usage"></a>

1. Add the block to your canvas.
2. Double-click or open its canvas to place and connect blocks that implement the desired functionality.
3. Provide an `Enable` input (for example, a `Logic Input` or a boolean signal) to control whether the subsystem runs. If you leave `Enable` disconnected the subsystem will run by default.
4. Connect outputs from internal blocks to the subsystem outputs (they will appear on the parent canvas when available).
5. Use the parent graph to route data into and out of the subsystem as needed.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use `Logic Input` to toggle the subsystem on/off manually during testing.
* Use `Rising Edge` if you want the subsystem to run only on the first activation rather than continuously.
* Place image acquisition blocks such as `Camera USB`, `Camera IP (ONVIF)`, or `Stream Reader` inside the subsystem when you want to enable/disable camera processing as a single feature.
* Combine with `Show Image`, `Image Logger`, or `Record Video` inside the subsystem to control visualization and saving only when the subsystem is enabled.
* Use `Data Read Local` / `Data Write Local` (or global equivalents) to exchange data between the parent graph and the enclosed subsystem when you need persistent state or cross-subsystem communication.
* For triggered or timed behavior, combine with blocks like `Delay Step` or `ON Delay` inside the subsystem to shape execution timing.
* Use the subsystem to encapsulate experimental pipelines (for example: preprocessing -> detector -> postprocessing). This lets you switch the entire pipeline on/off without re-wiring the main graph.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* Subsystem does not run: check the `Enable` input. If it is connected and False, the subsystem will be skipped.
* Expected outputs not visible: ensure the internal blocks are connected to produce outputs and the subsystem has been executed while enabled.
* Internal error state: open the subsystem canvas and inspect blocks for red/invalid indicators; fix configuration or missing inputs inside the canvas.
* Need to always reset outputs: place state-reset or initialization blocks inside the subsystem to ensure consistent default outputs when the subsystem is not running.


# Subsystem In

This block is used to import a value from a parent scenario into a subsystem (a nested scene) and make it available to blocks inside that subsystem. It acts as a numbered input port for the subsystem and automatically synchronizes with the parent scenario's inputs.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

This block does not declare any direct inputs (it receives data from the parent scenario rather than from local sockets).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`Generic` This output provides the value supplied from the parent scenario. Inside the subsystem, connect this output to any blocks that need the parent-provided data.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Title` The block displays a number that identifies its position among the subsystem inputs (e.g., 1, 2, 3).\
`Scene Index` Each block is assigned a scene index that determines which parent input it maps to. The displayed title reflects this index.

Note: The numbering is managed automatically. When you add, remove, or reorder subsystem input blocks, their displayed numbers update to match their order.

## ⚙️ Running mechanism <a href="#running-mechanism" id="running-mechanism"></a>

* When the subsystem runs, each `Subsystem In` block provides the value coming from the matching input of the parent scenario.
* In the main (parent) scenario, the corresponding parent input supplies the data that flows into the subsystem input.
* The block keeps its mapping in sync with the parent scenario so the subsystem receives the correct input even if inputs are added or reordered in the parent.
* The block supports being used in test or memory-backed contexts (the system adapts behavior depending on whether the subsystem is run standalone or embedded).

## 🎯 Features <a href="#features" id="features"></a>

* Automatic numbering and index mapping so subsystem inputs are easy to identify.
* Dynamic synchronization with parent inputs — socket types and order follow the parent scenario automatically.
* Works seamlessly with enabled/disabled subsystem modes (when you use the subsystem enabling mechanism, input behavior adapts accordingly).
* Designed to be lightweight and non-resizable for a compact flow-control layout.

## 📝 How to use <a href="#usage" id="usage"></a>

1. Place a subsystem and open its child scene.
2. Add one or more `Subsystem In` blocks inside the child scene. Each will appear with a number (1, 2, ...) that indicates which parent input it maps to.
3. In the parent scenario, provide data to the corresponding subsystem input sockets. That data will become available from the `Generic` output inside the subsystem.
4. Inside the subsystem, connect the `Generic` output to other blocks that need the parent-supplied value.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Use together with `Subsystem Out` to return results from the subsystem back to the parent scenario.
* Combine with `Subsystem Enabled` when you need conditional or toggleable subsystem execution; the input behavior adapts when the subsystem is enabled/disabled.
* If you want to persist or freeze an input value inside the subsystem, pair the `Generic` output with `Data Memory` to hold the value across runs.
* For debugging or to inspect what the parent is sending into the subsystem, feed the `Generic` output into `Debug Input` or display it with appropriate viewer/output blocks.
* To pass configuration values (numbers, text, or booleans) from the parent, use parent-side blocks like `Number Input`, `String Input`, or `Logic Input` and map them via the corresponding `Subsystem In` inside the child scene.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If a `Subsystem In` block shows an unexpected number, try adding or removing other subsystem inputs; numbering updates automatically.
* If the subsystem does not receive values, ensure the parent scenario provides data on the matching input socket and that the subsystem is properly added to the parent.
* If socket types need to change, the system will synchronize child ports to match the parent — reconnect flows after major changes to confirm correct behavior.


# Subsystem Loop

This function block repeats an internal workflow for each element of a list (or other sized collection). Use it when you want to process a series of items one-by-one inside a contained subsystem and gather results as lists.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

This function block does not have any input sockets at the top level.\
To provide data to the repeated workflow, use internal blocks inside the subsystem (for example `Subsystem In`).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

This function block does not expose direct output sockets at the top level.\
Results produced inside the repeated workflow are returned via internal blocks (for example `Subsystem Out`) and are collected as lists by this block.

## 🕹️ Controls <a href="#controls" id="controls"></a>

There are no visible parameter controls on the block itself. Interaction and configuration happen inside the contained subsystem workspace where you place input/output helper blocks:

* `Subsystem In` Use inside the subsystem to receive the current item (or shared inputs) for each loop step.
* `Subsystem Out` Use inside the subsystem to emit the per-step result that will be collected by the loop.

## 🎨 Features <a href="#features" id="features"></a>

* Iterative processing: runs the internal workflow once per element of the first sized collection you provide.
* Multi-list support: if multiple collections with the same length are supplied, they will be iterated in parallel. Use pairing helpers if you need explicit pairing.
* Aggregation: outputs produced by the internal workflow are collected as lists (one list per output) and returned after the loop finishes.
* Safe stop and status propagation: internal errors or invalid states are propagated so the parent scenario can react.

## ⚙️ Running mechanism (plain language) <a href="#how-it-works" id="how-it-works"></a>

* The block looks for the first collection-type input you provide (for example a list of images or numbers).
* For each element in that collection it opens the subsystem workspace and runs the contained workflow once using the current element.
* Any values you send outside the subsystem with `Subsystem Out` are appended to lists. After the loop finishes, you get one collected list per output.
* If you need to iterate multiple lists together, ensure they have the same length or pair them before feeding into the subsystem.

## 📝 How to use <a href="#usage" id="usage"></a>

1. Place the `Subsystem Loop` block in your scenario.
2. Double-click (or open) the subsystem workspace and add the following inside:
   * `Subsystem In` blocks to receive the per-step inputs you want to work on.
   * Your processing blocks (filters, detectors, calculations) to operate on each item.
   * `Subsystem Out` blocks to send results back to the loop aggregator.
3. If you want to iterate multiple lists together, prepare them using a list helper (for example `List Operations`) before feeding into the subsystem.
4. Run the scenario. The loop will run the internal workflow for every element and return aggregated lists of the outputs.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* To iterate multiple related lists in sync, prepare them with the `List Operations` block (use the pairing option if available) before feeding into the subsystem.
* Use `Get Element` inside the subsystem only if you need direct indexed access to lists supplied as shared inputs.
* For large image batches, combine `Batch Processing` and `Debatch` to reduce memory usage and keep per-step processing efficient.
* Use `Data Read Local` / `Data Write Local` (or Global variants) to share persistent data between the main scenario and the subsystem when a simple input/output connection is not convenient.
* If you need to run the same processing on individual images and then visualize results, connect a `Show Image` block after the loop outputs to preview collected images.
* When working with lists that may contain empty or missing entries, add an `Is None` check inside the subsystem to avoid unexpected errors.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* No iterations occur: Verify you supplied at least one collection-type input inside the subsystem (via `Subsystem In`).
* Outputs come back wrapped in single-element lists: Check how many `Subsystem Out` blocks are present and confirm they return the intended values per step.
* Different list lengths: If you expect multiple lists to iterate together, ensure they are the same length or pair them beforehand with `List Operations`.
* High memory usage: Prefer `Batch Processing` and `Debatch` or process smaller chunks inside the subsystem to reduce peak memory.


# Subsystem Out

This function block provides a single export point inside a subsystem. Use it to push data from the local subsystem context to the parent (main) scene or to other parts of your scenario. It is typically used in groups — each export point is mapped to a corresponding entry point in the parent scene.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`Generic` This input accepts any type of data you want to expose outside the subsystem (images, numbers, lists, dictionaries, etc.).

## 📤 Outputs <a href="#outputs" id="outputs"></a>

This function block does not produce direct outputs in its local view — instead it exposes the connected input to the parent context where corresponding entry points receive it.

## 🕹️ Controls <a href="#controls" id="controls"></a>

This block has no interactive controls.\
When used inside a subsystem, the block displays a small numeric title/index to indicate its export position relative to other export points.

## 🎨 Features <a href="#features" id="features"></a>

* `Expose Local Data` Allows any value from inside a subsystem to be made available to the parent scene.
* `Multiple Exports` Adding several such blocks in a subsystem creates multiple, ordered export channels. The displayed index helps you track the mapping order.
* `Type Syncing` The export point will adapt to the type of data connected so the parent entry point can match the same data type.
* `Safe Use in Main Scene` Placing the block in the main scene will forward its input value as a normal passthrough output for convenience during testing or when the subsystem is open.

## 📝 Usage Instructions <a href="#usage" id="usage"></a>

1. Add a subsystem container (use the `Subsystem` block) and work inside it.
2. Place one or more `Subsystem Out` blocks inside the subsystem where you want to export values.
3. Connect your internal processing results to the `Generic` input of each `Subsystem Out` block.
4. In the parent scene, add corresponding entry points to receive the exported data. The numeric label on each `Subsystem Out` helps match the correct parent input.

## 📊 Evaluation <a href="#evaluation" id="evaluation"></a>

When the scenario runs, this block exposes the current value present at its `Generic` input to the outer (parent) context. Multiple export blocks create an ordered list of exported values that the parent can consume in the same order.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Pair with `Subsystem In` to receive the exported values inside your main scene or another subsystem.
* Use `Subsystem Enabled` to control whether a subsystem is active and thus whether its exports are meaningful at a given time.
* If you need to expose multiple heterogeneous values through a single export channel, use `Mux` inside the subsystem to bundle values into one structured output, and use `Demux` or unpacking in the parent as needed.
* For persistent sharing or when implementing feedback loops, consider `Data Write Local` and `Data Read Local` to store and read data outside the regular direct export/import mapping.
* During debugging, route exported data to `Debug Input` or to logging/ export blocks such as `CSV Export` or `Image Logger` (for image data) to verify what the subsystem is producing.
* Keep export ordering explicit: add or remove export blocks in the subsystem carefully — the parent mapping follows the order of export points, and the numeric title will help you verify alignment.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* If the parent scene is not receiving the expected value, confirm that the number of export blocks inside the subsystem matches the number of corresponding entry points in the parent scene.
* If data types appear mismatched, ensure the value connected to the `Generic` input is the same type the parent expects (use `Data Type Converter` or `Mux` to normalize data before export).
* When experimenting, you can place the block in the main scene to preview outputs directly, then move it into a subsystem once the flow is stable.


# Subsystem

This function block acts as a self-contained workflow container. Use it to group related processing steps into a separate, editable workspace so complex scenarios stay organized and easier to manage.

## 📥 Inputs <a href="#inputs" id="inputs"></a>

`No default inputs`\
This block does not declare inputs by itself. Inputs can be created by adding input sockets inside the subsystem workspace and exposing them at the block boundary.

## 📤 Outputs <a href="#outputs" id="outputs"></a>

`No default outputs`\
This block does not declare outputs by itself. Outputs are defined by child blocks placed inside the subsystem and exposed as the subsystem outputs.

## 🕹️ Controls <a href="#controls" id="controls"></a>

`Open Subsystem Editor`\
Open the subsystem workspace to add or edit the contained workflow. (Typically available by double‑clicking the block or via the block context menu.)

`Expose Input`\
Create an input socket inside the subsystem and expose it on the block boundary so the parent flow can feed data in.

`Expose Output`\
Create an output socket inside the subsystem and expose it on the block boundary to send results back to the main flow.

`Run / Debug Inside`\
Run and debug the contained workflow independently from the main scenario to validate its behavior.

## 🎯 Key Features <a href="#features" id="features"></a>

* Encapsulation: Group preprocessing, detection, tracking, and export steps into a single reusable unit.
* Local workspace: The subsystem has its own editable canvas so you can design complex behavior without cluttering the main flow.
* Clear boundaries: Expose only the inputs and outputs you need, keeping the main graph simple.
* Isolated testing: Open and run the subsystem workspace to debug or validate sub-flows independently.
* Reusability: Build a tested subsystem once, then reuse it across multiple projects.

## ⚙️ How it runs <a href="#running-mechanism" id="running-mechanism"></a>

* When the scenario executes, the subsystem runs its internal workflow.
* Data flows into the subsystem through the input sockets you exposed inside it.
* Child blocks inside the subsystem compute their results in sequence.
* Outputs that you exposed inside the subsystem are collected and returned to the parent flow as the block outputs.
* If a child block inside the subsystem reports an error or an invalid state, the subsystem will reflect that status so you can locate and fix the issue inside the workspace.

## 📝 Usage Guidelines <a href="#usage" id="usage"></a>

1. Add a `Subsystem` block to your main flow to hold a complex sequence (acquisition → processing → export).
2. Open the subsystem workspace (`Open Subsystem Editor`) and add the needed blocks.
3. Inside the subsystem, expose only those inputs and outputs required by the parent flow.
4. Test the subsystem independently using the internal run/debug controls before integrating it into the main scenario.
5. Reuse and duplicate subsystems for similar tasks to speed up development.

## 💡 Tips and Tricks <a href="#tips-and-tricks" id="tips-and-tricks"></a>

* Group a complete camera pipeline inside a subsystem: use `Camera USB` or `Camera IP (ONVIF)` → preprocessing blocks like `Image Resize` / `Blur` → detectors such as `Object Detection` or `Find Object` → visualization with `Draw Detections` → storage with `Image Logger` or `Image Write`.
* For tracking flows, place `Object Detection` (or `Object Detection - Custom`) and `Object_Detection_Tracker` inside a single subsystem so tracking logic and post-processing stay together.
* Encapsulate heavy AI steps like `Super Resolution`, `Depth Estimation (DepthAny. V2)`, or `Background Removal (RMBG-1.4)` inside subsystems to make it easy to enable/disable or replace models without disturbing the main flow.
* Use subsystems to implement multi-stage analysis: e.g., ROI selection with `Image ROI Select` → shape analysis with `Find Contour` / `Measure Position Distance` → result formatting with `Data to JSON`.
* When building intersection or traffic analytics, keep detection and tracking in one subsystem and aggregation/export (for example the `Traffic Intersection Analysis` block or `CSV Export`) in another to separate concerns.
* For data passing and local state, combine subsystems with `Data Write Local` and `Data Read Local` to maintain local memory without polluting global flow.
* When troubleshooting, open the subsystem and run only its contents to quickly identify which internal block is causing invalid or unexpected outputs.

## 🛠️ Troubleshooting <a href="#troubleshooting" id="troubleshooting"></a>

* `No outputs appearing`\
  Confirm you have exposed outputs inside the subsystem. If nothing is exposed, the block will not return data to the parent flow.
* `Inputs not received`\
  Make sure you created and exposed input sockets inside the subsystem and that the parent flow is connected to those exposed inputs.
* `Child block reports invalid`\
  Open the subsystem workspace and inspect child blocks. Errors inside the subsystem will surface there—fix the child block configuration or replace the problematic block.
* `Performance issues`\
  If the subsystem contains heavy processing (AI models, super-resolution, large batch operations), consider profiling inside the subsystem and moving non-essential processing to separate, on-demand subsystems.
* `Need to reuse or version`\
  Duplicate the subsystem and keep one testable copy. This makes it safe to try different parameters or models without affecting the production flow.


# logic




---

[Next Page](/llms-full.txt/1)