65 lines
3.9 KiB
Markdown
65 lines
3.9 KiB
Markdown
# VTScada-HistoricalTools
|
|
|
|
This is a set of tooling to perform useful operations of queryign and moving Historical data into ClearSCADA
|
|
|
|
Currently supports querying data from the following sources, and their requirements
|
|
* VTScada - REST
|
|
* ClearSCADA - Raw Historic Files
|
|
* AVEVA (Wonderware) Historian - InSQL
|
|
|
|
The primary function of this tooling is to query a set of tags for a specified date range, compress and process those values as required, and move it into a format which can be easily imported into VTScada.
|
|
|
|
## Setup
|
|
|
|
### TagMap file
|
|
|
|
The TagMap file is a CSV file of which tags to query on the existing system. They are queried with the following paramaters.
|
|
* Interval - time in seconds to query the data point
|
|
* Deadband - values within this deadband before scaling will be elimianted
|
|
* Scale Factor - scaling factor to apply to existing data before compression
|
|
* Precision - the final result value will be rounded to this many decimal places
|
|
|
|
For all values, values will either be working from a set of all data (for example, ClearSCADA), or querieid at the required interval (for example, VTScada REST). For the former, values within the interval will be deleted after compression.
|
|
|
|
For boolean values, they will be **sampled** at the interval specified and compressed by eliminating values which stay the same. To ensure the precision of transitions wanted is kept, a high precision such as 1 second sampling is recommended here
|
|
|
|
For integer/analog values, they will be **averaged** at the interval specified, and compressed by eliminating values within the deadband, then scaled, then rounded to the required precision. A lower precision of sampling is recommended here, perhaps 5 seconds for high-value points, and 30-60 seconds for low value points.
|
|
|
|
### Setup TOML
|
|
|
|
This file contains configuration on how to connect to the required existing interfaces and locations of input and the output data.
|
|
|
|
### Python
|
|
|
|
This system requires Python installed and run on the system which can target existing interfaces. Several libraries will need to be installed by pip.
|
|
|
|
## VTScada REST
|
|
|
|
This is a method of moving VTScada data into VTScada data. Scenarios where this could be useful:
|
|
* Moving tag data from Analog Status / Digital Status to the modern IO tags
|
|
* Tag data sizes which have gotten out of hand
|
|
* A site has been recommissioned with a new set of tags and data needs to be imported
|
|
|
|
## ClearSCADA - Raw Historic Files
|
|
|
|
In places where targetting a live ClearSCADA system with SQL queries is challenging, ClearSCADA uses a file-based Historian and provides a utility which converts these HRD files into CSV data.
|
|
|
|
For each week of each data point, a separate CSV file of data is created from the
|
|
|
|
Files are generally stored:
|
|
```C:\ProgramData\Schneider Electric\ClearSCADA\Database\HisFiles```
|
|
|
|
Each directory contains a Historic XXXXXX directory where XXXXX is the Unique ID of the datapoint padded with a file WKYYYYYY where YYYYYY is the number of weeks since January 1, 1601 (yes, really).
|
|
|
|
These tools will conver the user start time and end time in a way that will only process the found and required HRD files at a time. This can *greatly* expand the amount of data in the system, it is strongly recommended to have a lot of free space left during queries.
|
|
|
|
|
|
### Setup ClearSCADA Config
|
|
|
|
Key Point: In the tags list file, the Source Name field is the unique identifier for the tag name to query. In VTScada this can be something like ```temp\old_value1```. In ClearSCADA, it will be the unique point ID, ex. ```005152```. The leading zeroes can be left out as the script will pad them in front of the integer to determine the correct path.
|
|
|
|
## AVEVA (Wonderware) Historian - InSQL
|
|
|
|
Prerequisite: Install the Microsoft ODBC Driver, and assign the correct name to the driver in the *setup.toml* file.
|
|
|
|
Data can be queried from an AVEVA Historian through an ODBC connection. The Historian provides an interface to SQL, InSQL, which can be used to create data queries. |