ValMetTelem - Revision history

NebarnixWikiSysop: /* CRC Inspection */

2016-04-01T01:12:28Z

CRC Inspection

2016-03-31T18:08:17Z

2016-03-31T18:06:28Z

2016-03-31T18:04:39Z

updated code

2016-03-31T15:39:58Z

2016-03-31T15:33:11Z

2016-03-31T15:32:30Z

2016-03-31T15:31:39Z

2016-03-31T15:30:34Z

2016-03-31T15:30:23Z

@@ Line 47: / Line 47: @@
 I had no idea if these two trailing bytes were really CRC or not. It appeared to be, as identical packets had identical bytes at the end, without exception.
-The CRC is not standard, but is close. I used the following website http://www.sunshine2k.de/coding/javascript/crc/crc_js.html and just tried the whole dropdown list until I landed on CRC16-genibus. If my bytes or polarity was off, this would not work, so this verifies the byte alignment and polarity of our bits. I actually didn't really think I would ever find the algorithm ;)
+The CRC is not standard, but is close. I used the following website http://www.sunshine2k.de/coding/javascript/crc/crc_js.html and just tried the whole dropdown list until I landed on CRC16-genibus. If my bytes or polarity were off, this would not work, so this verifies the byte alignment and polarity of our bits. I actually didn't really think I would ever find the algorithm ;)
 The CRC-16 settings are as follows:<br />

@@ Line 370: / Line 370: @@
 == Packet Dumps ==
-Here are three dumps consisting of 10MB of recorded data from each of the three frequencies. The data has been filtered by CRC, and totally empty packets have been removed.
+Here are three dumps consisting of 10MB of recorded data from each of the three frequencies. The data has been filtered by CRC, and totally empty packets have been removed. The next step would be to color code 0x8E faded gray and filter out packets that don't contain messages (3rd byte == 0x00)
 <pre>

@@ Line 8: / Line 8: @@
 A demodulator and bit-sync was written in GRC. The deviation was seen to be 4800Hz using baudline, and a quick quadrature demod produced the following waveform, from which we can measure a bit period and see that this stream is 4800 symbols per second. Using this information I was able to construct a polyphase clock recovery pathway in GRC and recover the data stream bits.
 [[File:MetroDemodPacket.png|1200px]]
@@ Line 269: / Line 271: @@
 %% Spit out frames as hex
 for idx=1:size(minorFrames,1)
-    fprintf('%0.2X',minorFrames(idx,:));
+    fprintf('%0.2X ',minorFrames(idx,:));
     fprintf('\n');
 end
-%% Verify with CRC-16 and pull valid bytes into new array
+%% Optional, kil empty packets by invalidating the CRC
 goodData = 0;
 idx2 = 1;
 minorFramesValid = zeros(1,29);
 for idx=1:size(minorFrames,1)
      if((minorFrames(idx,30)*2^8+minorFrames(idx,31)) == crc16(minorFrames(idx,3:29)))
          goodData = goodData+1;
@@ Line 286: / Line 299: @@
 end;
 fprintf(['Valid packets: ' num2str(size(minorFramesValid,1)) ' (' num2str(goodData/size(minorFrames,1)*100,3) '%% valid)\n' ]);
 </pre>

@@ Line 47: / Line 47: @@
 The CRC is not standard, but is close. I used the following website http://www.sunshine2k.de/coding/javascript/crc/crc_js.html and just tried the whole dropdown list until I landed on CRC16-genibus. If my bytes or polarity was off, this would not work, so this verifies the byte alignment and polarity of our bits. I actually didn't really think I would ever find the algorithm ;)
-The CRC-16 settings are as follows:
+The CRC-16 settings are as follows:<br />
-Polynomial: 0x1021
+Polynomial: 0x1021<br />
-Initial Value: 0xFFFF
+Initial Value: 0xFFFF<br />
-Final XOR value: 0xFFFF
+Final XOR value: 0xFFFF<br />
 ==== CRC Matlab Code ====

@@ Line 345: / Line 345: @@
 * Three bytes of status or mode -> the third of which seems to contain a bit that indicates that there IS a message present.
 * 24 bytes of message structure which appears to be empty in most packets (very strange!)
-* Two byte CRC maybe?
+* Two byte CRC-16 (GENIBUS format)
 * An end of packet marker 0x7E (also weird!)
 * another byte of transitions 0XAA, some sort of turn-off delay for the radio to avoid clipping off the end of a packet.

@@ Line 38: / Line 38: @@
 Confirmation of data structure made by grabbing a second packet:
 AA AA 7E 00 00 00 8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E 97 80 7E AA
 </pre>
 === CRC Inspection ===
-I had no idea if this was really CRC or not. It appeared to be, as identical packets had identical bytes at the end, without exception.
+I had no idea if these two trailing bytes were really CRC or not. It appeared to be, as identical packets had identical bytes at the end, without exception.
 The CRC is not standard, but is close. I used the following website http://www.sunshine2k.de/coding/javascript/crc/crc_js.html and just tried the whole dropdown list until I landed on CRC16-genibus. If my bytes or polarity was off, this would not work, so this verifies the byte alignment and polarity of our bits. I actually didn't really think I would ever find the algorithm ;)

@@ Line 38: / Line 38: @@
 Confirmation of data structure made by grabbing a second packet:
 AA AA 7E 00 00 00 8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E 97 80 7E AA
 </pre>
@@ Line 245: / Line 287: @@
 == Possible Packet Structure Dissection==
-By looking at the following chunk of data we can assume some things:
+By looking at the following chunk of CRC validated data we can assume some things:
 <pre>
 00 00 8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E8E 97 80
@@ Line 305: / Line 347: @@
 * another byte of transitions 0XAA, some sort of turn-off delay for the radio to avoid clipping off the end of a packet.
 == Data Contents ==
 Attempts to reverse engineer the packet structure are still in work. Multiple patterns have been already noted, but as most packets are actually empty, more data is required than would be assumed at first to decode what these messages might contain.

← Older revision		Revision as of 15:30, 31 March 2016
Line 9:		Line 9:
	A demodulator and bit-sync was written in GRC. The deviation was seen to be 4800Hz using baudline, and a quick quadrature demod produced the following waveform, from which we can measure a bit period and see that this stream is 4800 symbols per second. Using this information I was able to construct a polyphase clock recovery pathway in GRC and recover the data stream bits.		A demodulator and bit-sync was written in GRC. The deviation was seen to be 4800Hz using baudline, and a quick quadrature demod produced the following waveform, from which we can measure a bit period and see that this stream is 4800 symbols per second. Using this information I was able to construct a polyphase clock recovery pathway in GRC and recover the data stream bits.

−	[[File:MetroDemodPacket.png\|~~800px~~]]	+	[[File:MetroDemodPacket.png\|1200px]]

@@ Line 4: / Line 4: @@
 [[file:MetroPacketsWaterfallView.png | 400px]]
 ==Data Data what is the Data!==
-Using GNU-radio and baudline, I discovered that these FSK signals have a deviation of 4800Hz, and a bitrate of also 4800Hz. I constructed a demodulator and bitsync in GNUradio, and discovered the following packet structure. It is  easy to see that this is some sort of manchester encoding as there is a transition during every 2 bis and there are no consecutive bits within the groups.
+After bit-sync, we can look at the following packet structure. It is  easy to see that this is some sort of manchester encoding as there is a transition during every 2 bis and there are no consecutive bits within the groups.
 <pre>