Hi,

I would like to ask for your opinion, find some suggestions on what to use (if there is any existing library I don’t know about), and how to approach it.

I am about to build a new product for parsing Comtrade .cfg and .dat files (there are two more files, but are not mandatory). COMTRADE (Common format for Transient Data Exchange for power systems) is a file format for storing oscillography and status data related to transient power system disturbances .

For better understanding, and if interested into the subject, you can take a look into this to get better understanding
http://dfv-technologie.com/doc_technique_dfv/perturbographie_qualimetrie/Tutorial%20COMTRADE.pdf

For others TLDR; you have .CFG files which describes a structure of .dat files. We’re talking about a mixture of ASCII and BINARY data and so on. Once you create a parser to understand the rules .cfg file define, you can go and parse binary data (.DAT) file. Each event has 1 .cfg and 1 .dat file. Later on I will need to do some correlations between 2 events and so on.

really simplified CFG example

EXAMPLE STATION NO. 1 DAU 32 N,32
15,7A,8D
1,161KV 01-G BUS POTENTIAL,kV, 0.479615,0.0,0.0,-511,511
2,161KV 02-G BUS POTENTIAL,kV, 0.479615,0.0,0.0,-511,511
3,161KV 03-G BUS POTENTIAL,kV, 0.479615,0.0,0.0,-511,511
4,161KV 01 CURRENT, A, 43.187479,0.0,0.0,-511,511
5,161KV 02 CURRENT, A, 43.187479,0.0,0.0,-511,511
6,161KV 03 CURRENT, A, 43.187479,0.0,0.0,-511,511
7,161KV RES CUR., A, 43.187479,0.0,0.0,-511,511
1,PCB 1216 EXAMPLE,1
2,PCB 1226 EXAMPLE,1
3,PCB 1236 EXAMPLE,1
4,161KV CAR. REC,1
5,161KV BUS POTENTIAL UV,1
6,161KV BUS POTENTIAL UF,1
7,EXTERNAL START,1
8,DATA CHECKSUM ERROR N/C,0
60
1
5184,47
01/18/1992,10: 2: 5.136
01/18/1992,10: 2: 5.215
ASCII
####################################################

really simplified DAT example

1,192,252,-233,-19,0,0,0,0,1,1,0,1,1,1,1,0
2,384,243,-243,0,0,0,0,0,1,1,0,1,1,1,1,0
3,576,232,-252,20,0,0,0,0,1,1,0,1,1,1,1,0
4,768,220,-260,40,0,0,0,0,1,1,0,1,1,1,1,0
5,960,207,-267,60,0,0,0,0,1,1,0,1,1,1,1,0
6,1152,192,-273,79,0,0,0,0,1,1,0,1,1,1,1,0
7,1344,176,-276,98,0,0,0,0,1,1,0,1,1,1,1,0
8,1536,160,-279,117,0,0,0,0,1,1,0,1,1,1,1,0
9,1728,142,-280,135,0,0,0,0,1,1,0,1,1,1,1,0
10,1920,124,-279,153,0,0,0,0,1,1,0,1,1,1,1,0
11,2112,105,-277,171,0,0,0,0,1,1,0,1,1,1,1,0
12,2304,85,-274,187,0,0,0,0,1,1,0,1,1,1,1,0
13,2496,65,-269,203,0,0,0,0,1,1,0,1,1,1,1,0
14,2688,45,-263,218,0,0,0,0,1,1,0,1,1,1,1,0
15,2880,25,-255,231,0,0,0,0,1,1,0,1,1,1,1,0
16,3072,5,-246,243,0,0,0,0,1,1,0,1,1,1,1,0
17,3264,-16,-235,253,0,0,0,0,1,1,0,1,1,1,1,0
18,3456,-36,-223,262,0,0,0,0,1,1,0,1,1,1,1,0
19,3648,-56,-210,269,0,0,0,0,1,1,0,1,1,1,1,0
20,3840,-76,-196,276,0,0,0,0,1,1,0,1,1,1,1,0
####################################################

This example above is ASCII type, mine are BINARY

Would you go and implement this on your own from the ground up or you would use some specific (existing) parsing library? I am asking if anybody knows of any library that might help me with parsing itself or if you don’t see this as an issue and would just write your own.

The other still “open” issue on this product is its integration into the Microsoft domain. I assume this shouldn’t be an issue anymore. MS LDAP and other keywords are probably well-supported by now. If you have any thoughts to share more than welcome.

Thanks for your opinion in advance.