---
alias: user-guide-driver-csv
tags:
- connect iot
description: "This documentation details a CSV file driver for IoT devices, enabling monitoring of comma-separated data files with configurable settings and"
---

# Connect IoT - CSV File Driver

A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. A CSV file stores tabular data (numbers and text) in plain text. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format.

## Data Types

These are the supported specific data types:

| Name    | Description      |
| ------- | ---------------- |
| String  | String Value     |
| Boolean | Boolean Value    |
| Integer | Integer value    |
| Decimal | Decimal value    |
| Object  | Structured value |

## Protocol Parameters

The protocol supports the following parameters, used define the communication parameters, as well as the expected behavior of the driver:

| Name                               | Type    | Possible Values  | Default Value | Description                                                  |
| ---------------------------------- | ------- | -----------------| ------------- | ------------------------------------------------------------ |
| path                               | String  |                  | ""            | Path of the location to monitor for changes on the files. If no value is provided, no watcher is created, so no events will be triggered.|
| fileMask                           | String  |                  | "**/*.csv"    | Glob pattern to use for the watcher to identify the files to handle. Use a tool like <https://globster.xyz/> to try a valid value to use.|
| hasHeader                          | Boolean |                  | True          | Do the csv files have header? |
| delimiter                          | String  |                  | ","           | What is the separator of the fields? |
| initialLinesToIgnore               | Integer |                  | 0             | Number of lines to ignore from the beginning of the file |
| commentChar                        | String  |                  | ""            | If set, all text after the character will not be handled |
| numberOfLines                      | Integer |                  | 1             | Number of lines of each file to read per round |
| skipEmptyLines                     | Boolean |                  | True          | Should the parser ignore empty lines? |
| archivePath                        | String  |                  | ""            | Directory planned to use for archiving the files after processing. This value can later be used as a variable in the available operations to execute.|
| watcherType                        | Enum    | Chokidar<br>NSFW | Chokidar      | Type of watcher to use. Depending on the selection, different settings are necessary. Chokidar is the best overall option, but CPU and Memory heavy and slower to start when many files are present. NSFW is better other scenarios. Test both to determine the best option. |
| ignoreInitial                      | Boolean |                  | False         |  Flag to define if, during startup/restart, the files that were created, changed, deleted should be processed or ignored |
| pollingInterval                    | Integer |                  | 100           | Interval of file system polling (in milliseconds) |
| alwaysStat                         | Boolean |                  | True          | Always get additional attributes (size, timestamps, etc) of the file that was identified by the watcher. Will require additional operating system resources.|
| depth                              | Integer |                  | 0             | Number of sub directories to watch. The higher the number, more memory/cpu/time will be required for watchers. |
| awaitWriteFinish                   | Boolean |                  | True          | Trigger watcher events only when the file finishes writing. |
| awaitWriteFinishStabilityThreshold | Integer |                  | 2000          | Amount of time in milliseconds for a file size to remain constant before emitting its event. |

### Protocol Parameters specific for Chokidar watcherType

Chokidar watcher type has some settings that are only known to it. If using the other type(s), these settings will not have any effect.

| Name                         | Type    | Possible Values               | Default Value | Description                                                  |
| ---------------------------- | ------- | ----------------------------- | ------------- | ------------------------------------------------------------ |
| watcherMode                  | Enum    | FileSystemEvents<br />Polling | Polling       | To successfully watch files over a network (and in other non-standard situations), it is typically necessary to use Polling, however it could lead to high CPU utilization. FileSystemEvents is the most efficient method for monitoring local files. |
| pollingBinaryInterval        | Integer |                               | 300           | Interval of file system polling for binary files (in milliseconds) |
| awaitWriteFinishPollInterval | Integer |                               | 100           | File size polling interval                                   |
| atomic                       | Integer |                               | 100           | Automatically filters out artifacts that occur within the defined value (in milliseconds) when using editors that use 'atomic writes' instead of writing directly to the source file. If defined to 0, this function is disabled. |

## Extended Parameters

As with other protocol drivers, some extra parameters are necessary to provide more context to the behavior expected.

### Property

| Name         | Type | Possible Values                                              | Default Value | Description                                                  |
| ------------ | ---- | ------------------------------------------------------------ | ------------- | ------------------------------------------------------------ |
| propertyType | Enum | FileContent<br>FileName<br />RelativePath<br />FullPath<br />Size<br />IsFile<br />IsDirectory<br />AccessTime<br />CreationTime<br />UpdateTime<br>EndOfFile | FileContent | Provides a meaning of the property, and will receive the value depending of it when specific events occur.<br>`FileContent` Representation of a field in the file<br />`FileName` Name of the file being handled<br />`RelativePath` Relative path of the file being handled<br />`FullPath` Full path of the file being handled<br />`Size` Size of the file<br />`IsFile` Flag indicating if the handle is a file<br />`IsDirectory` Flag indicating if the handle is a directory<br />`AccessTime` Date and Time when the file was last accessed<br />`CreationTime` Date and Time when the file was created<br />`UpdateTime` Date and Time when the file was last updated<br>`EndOfFile` Flag indicating the end of file was reached |
| identifierType | Enum | Name<br>Index<br>None | Name | Way to identify the field data (defined in the Property Device Id. Only applicable when propertyType is `FileContent`|

### Event

| Name         | Type | Possible Values                           | Default Value | Description                                                  |
| ------------ | ---- | ----------------------------------------- | ------------- | ------------------------------------------------------------ |
| EventTrigger | Enum | NewFile<br />DeletedFile<br />ChangedFile<br>NewData | NewData | Indicates the Operating System action that will trigger the event.<br />`NewFile` When a new file is detected <br>`DeletedFile` When a file is deleted <br>`ChangedFile` When a file is changed<br>`NewData` When the file is created/changed, trigger an event with a number of lines (configurable) |

### Command

| Name        | Type | Possible Values                                              | Default Value | Description                                                  |
| ----------- | ---- | ------------------------------------------------------------ | ------------- | ------------------------------------------------------------ |
| commandType | Enum | Exists<br />Copy<br />Rename<br />Delete<br />Move<br />CreateFile<br />ReadFile<br />CreateDirectory<br />EmptyDirectory<br />Mount<br>MoreDataRequest | MoreDataRequest | Indicated the type of command that will be executed.<br />`Exists` Checks if a file or directory exists<br />`Copy` Copies a file or entire directory from one location into another<br />`Rename` Renames a file or directory<br />`Delete` Deletes a file or directory<br />`Move` Moves a file or directory<br />`CreateFile` Create or append data into a file<br />`ReadFile` Reads the entire content of an existing file<br />`CreateDirectory` Creates a new directory<br />`EmptyDirectory` Deletes all files and subdirectories of a directory<br />`Mount` Mounts a network drive location into a local path<br />`MoreDataRequest` After Data is processed, request for the next chunk.<br />`ListFiles` List all files in a directory. |

## Internal Driver Interface

This protocol driver provides a set of interfaces ready to be used to ease the usability within the Controller Workflows. This section is intended to detail them. Although most if not all of these interfaces have specifically designed tasks to use them, the user is free to redefine them in the Driver Definition entity.

### Properties

| Name         | PropertyType   | Data Type | Description                                                  |
| ------------ | -------------- | --------- | ------------------------------------------------------------ |
| FileName     | `FileName`     | String    | Name of the file identified in the watched directory. Will include the extension, but not the directory path. |
| FullPath     | `FullPath`     | String    | File name and the full path where it is located in the local system. |
| FileSize     | `Size`         | Integer   | Size of the file                                             |
| EndOfFile    | `EndOfFile`    | Boolean   | End of file reached                                          |
| ExampleField | `FileContent`  | String   | Example of a field of the Csv File identified by Header name |
| ExampleField2| `FileContent`  | String  | DExample of a field of the Csv File identified by index (2nd)|

### Events

| Name          | EventTrigger  | Properties updated                                           | Description                                                  |
| ------------- | ------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| OnNewFile     | `NewFile`     | `FileName`<br />`FullPath`<br /> `FileSize`<br /> | When the watcher detects a file that was not previously identified appears, will raise this event.<br />The timing when it will be triggered may depend on some settings, like `watcherMode`, `awaitWriteFinish`, `awaitWriteFinishStabilityThreshold`  and others. |
| OnFileChanged | `ChangedFile` | `FileName`<br />`FullPath`<br /> `FileSize`<br /> | When the watcher detects a file that was already identified (and triggered the `OnNewFile`event), and it was changed in some way, will raise this event.<br />The timing when it will be triggered may depend on some settings, like `watcherMode`, `awaitWriteFinish`, `awaitWriteFinishStabilityThreshold`  and others. |
| OnFileDeleted | `DeletedFile` | `FileName`<br />`FullPath`<br />`FileSize`<br /> | When the watcher detects a file that was previously identified and was now deleted (or moved somewhere), will raise this event. |
| OnDataAvailable | `NewData` | `FullPath`<br />`ExampleField`<br />`ExampleField2`<br /> `EndOfFile` | Triggered when data inside the file was read and parsed |

### Commands

All paths of the commands listed in this section can use the following tokens to simplify usability:

| Name         | Description                                                  | Example                                                      |
| ------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| `${Path}`    | Replace the token with the `path` setting defined during setup | `${path}\file.csv` becomes `c:\watchedDirectory\file.csv`    |
| `${Archive}` | Replace the token with the `archive` setting defined during setup | `${archive}\newFile.csv` becomes `c:\archive\file.csv`       |

| Name            | CommandType       | Parameters / Type / Default Value                            | Description                                                  |
| --------------- | ----------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| MoreDataRequest | `MoreDataRequest` | `path` / String / ""                                         | After Data is processed, request for the next chunk          |
| FileOrDirectoryExists | `Exists`    | `path` / String / ""                                         | Checks the `path` location exists                            |
| CopyFileOrDirectory   | `Copy`      | `source` / String / ""<br />`destination` / String / ""<br />`overwrite` / Boolean / false<br />`preserveTimestamps` / Boolean / false<br />`recursive` / Boolean / true<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to copy the `source` location (file or directory) into the `destination` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time.<br />If it is a directory and `recursive` is set to true, all subdirectories will also be copied. |
| RenameFileOrDirectory | `Rename`    | `source` / String / ""<br />`destination` / String / ""<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to rename the `source` location (file or directory) into the `destination` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time.<br /> |
| MoveFileOrDirectory   | `Move`   | `source` / String / ""<br />`destination` / String / ""<br />`overwrite` / Boolean / false<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to move the `source` location (file or directory) into the `destination` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time.<br /> |
| DeleteFileOrDirectory | `Delete` | `path` / String / "" / <br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Deleted the `path` location (file or directory). If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time. |
| CreateFile      | `CreateFile`      | `path` / String / ""<br />`content` / String / ""<br />`encoding` / String / "utf8"<br />`mode` / Integer / 0o666<br />`flag` / String / "w"<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to write into a file located in the `path` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time.<br /><br />The `mode` will define the file attributes / permissions using unix format, but in decimal.<br />The `flag` determines how the file will be accessed (write, rewrite, append, etc). The supported flags are described in <https://nodejs.org/api/fs.html#fs_file_system_flags.> |
| ReadFileContent        | `ReadFile`        | `path` / String / ""<br />`encoding` / String / "utf8"<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to read the contents of the file in the `path` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time. |
| CreateDirectory | `CreateDirectory` | `path` / String / ""<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to create a new directory in the `path` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time. |
| EmptyDirectory  | `EmptyDirectory`  | `path` / String / ""<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to delete all files and subdirectories of  `path` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time. |
| ListFiles       | `ListFiles`       | `path` / String / ""<br />`attempts` / Integer / 1<br />`sleepBetweenAttempts` / Integer / 1000 | Tries to list all files in the `path` location. If it fails, will retry `attempts` times, waiting `sleepBetweenAttempts` milliseconds each time. |

## Driver Extensions

This protocol driver provides a set of extension that allow the customization to perform actions that are independent of the Driver Definitions associated. This means you can, using customization, get/set node values without needing to describe them.

> This is particularly useful when accessing equipment data that is shared/too complex to access using the regular mechanisms. Also useful when needing to access data whose naming convention may vary from device to device.

### Objects

The objects are structures that are either passed as parameters or returned as result values.

#### RegisterHandler

| Name | Type   | Description                                                  |
| ---- | ------ | ------------------------------------------------------------ |
| type | String | Id of the event to register a notification. Example: `connect.iot.driver.fileBased.event.ChangedFile_1234567891011`|

#### UnregisterHandler

| Name | Type   | Description                                                  |
| ---- | ------ | ------------------------------------------------------------ |
| type | String | Id of the event to unregister notifications. Example: `connect.iot.driver.fileBased.event.ChangedFile_1234567891011` |

#### ExecuteCommand

| Name       | Type        | Description                                                  |
| ---------- | ----------- | ------------------------------------------------------------ |
| command    | Json object | Includes the command name, device id and parameters types    |
| parameters | Json object | Parameter names and values.                                  |

### Methods

#### Register Event Notification

Notifies the protocol driver to raise the occurrences of a specific event when it occurs.

`void connect.iot.driver.fileBased.registerEvent(data: RegisterHandler)`

!!! note
    The events will be published to the listeners `connect.iot.driver.fileBased.` appended with the name of the event registered and a random id. Example: `connect.iot.driver.fileBased.event.ChangedFile_1234567891011`

#### Unregister Event Notification

Notifies the protocol driver to stop raising the occurrences of a specific event when they occur.

`void connect.iot.driver.fileBased.unregisterEvent(data: UnregisterHandler)`

#### Example

Notify `connect.iot.driver.fileBased.unregisterEvent`

```json
{
  "name": "ChangedFile_1234567891011"
}
```

### Execute Command

Executes a custom command.

`any connect.iot.driver.fileBased.executeCommand(data: ExecuteCommand)`

#### Example

SendRequest `connect.iot.driver.fileBased.executeCommand`

```json
{
    "command": {
        "name": "FileOrDirectoryExists",
        "deviceId": "Exists",
        "extendedData": { "commandType": "Exists" },
        "parameters": [
            { "name": "path", "dataType": "String", "deviceType": "String" },
        ],
    },
    "parameters": {
        "path": "c:/temp/file.csv",
}
```