ParseBinary Issue

Topics: Parsers
Sep 3, 2013 at 12:34 PM
Edited Sep 3, 2013 at 1:08 PM
I have attempted to use parse binary to parse a serial stream with no avail. I have identified the problem as a sync pattern issue.
            //IEnumerable<byte> bytes = new byte[] { 15, 175, 3, 32, 0};      // Works
            IEnumerable<byte> bytes = new byte[] { 6, 15, 175, 3, 32, 3 };    // Fails
                       
            byte[] someBytes = new byte[] { 15, 175, 3, 32 };
            var someString = Encoding.UTF8.GetString(someBytes);

            var parsed = bytes.ParseBinary(parser =>
                from next in parser
            //Both of these produce the same result:
            let sync = parser.HexString(4).Where(x => x.Equals("0F-AF-03-20"))
            //let sync = parser.String(Encoding.UTF8, 4).Where(value => value == someString)

            // ** THIS WILL ALWAYS WORK ! **
            //let sync1 = parser.HexString(1).Where(value => value == "0F")
            //let sync2 = parser.HexString(1).Where(value => value == "AF")
            //let sync3 = parser.HexString(1).Where(value => value == "03")
            //let sync4 = parser.HexString(1).Where(value => value == "20")                      
            // **********************
            let id = from data in next.Exactly(1)
                    select data                               

            // ** THIS WILL WORK ! **
            //select from _a in sync1      
            //       from _b in sync2
            //       from _c in sync3
            //       from _d in sync4
            // **********************
            select from _ in sync
                    from i in id       
                    select i
            );

           foreach (var value in parsed)
               Console.WriteLine(value);
As you can see from the code, the adding the byte at the leading edge of the sequence causes the Parser to not parse. If i remove the byte, then the parser works as desired. The problem that I have with this is that I am reading a byte stream that most likely will not start with the sync pattern and can potentially loose sync. I have been able to "workaround" the issue by using 4 sync parameters, but this is not a long term possibility as the sync length/data will change. Any suggestions?

EDIT: Dug through the code and seems the definition:
protected IParser<byte, string> HexString(int length)
will seek through the next "length" bytes to find the HexString...

Any way to get the following code to work with the existing framework?
 IEnumerable<byte> bytes = new byte[] { 15, 175, 3, 32, 0, 3, 15, 175, 3, 32, 22, 123};    // Fails on second message,  Returns 0
 IEnumerable<byte> bytes = new byte[]  { 15, 175, 3, 32, 0, 3, 4, 23, 24, 15, 175, 3, 32, 22, 123};  // Works Returns 0, 22
Coordinator
Sep 3, 2013 at 2:12 PM
Edited Sep 3, 2013 at 2:30 PM
Hi,

String and HexString have the semantics of Exactly. Furthermore, grammars are unambiguous and greedy by default, so after your query fails on the first byte it skips ahead the full 4 bytes before trying again.

There are two ways that you can fix your query:
  1. My recommendation is to use an n-ary binary comparison. It may be more semantic and efficient than #2.
  2. Apply Ambiguous(untilCount: 1) to the final grammar. Essentially, it reverses the effects of the exactly behavior that is introduced by String and HexString. Note that you'll also probably want to add SelectMany to the outer query to flatten the results.
Examples:

#1 (Recommended)
using System;
using System.Linq;
using Rxx.Parsers.Linq;

namespace RxLabs.Net45
{
    class ParseBinaryAmbiguityLab
    {
        internal static void Main()
        {
            //var bytes = new byte[] { 15, 175, 3, 32, 0 };      // Works
            var bytes = new byte[] { 6, 15, 175, 3, 32, 3 };    // Works

            var matchBytes = new byte[] { 15, 175, 3, 32 };

            var parsed = bytes.ParseBinary(parser =>
                    from next in parser
                    let sync = matchBytes.Select(b => parser.Byte.Where(b_ => b_ == b)).All()
                    let id = next.Exactly(1)
                    select from _ in sync
                           from i in id
                           select i
            );

            foreach (var value in parsed)
                Console.WriteLine(value);

            Console.ReadLine();
        }
    }
}
#2 (Not recommended, if you can avoid it)
using System;
using System.Linq;
using Rxx.Parsers.Linq;

namespace RxLabs.Net45
{
    class ParseBinaryAmbiguityLab
    {
        internal static void Main()
        {
            //var bytes = new byte[] { 15, 175, 3, 32, 0 };      // Works
            var bytes = new byte[] { 6, 15, 175, 3, 32, 3 };    // Works

            var parsed = bytes.ParseBinary(parser =>
                    from next in parser
                    let sync = parser.HexString(4).Where(x => x.Equals("0F-AF-03-20"))
                    let id = next.Exactly(1)
                    select (from _ in sync
                            from i in id
                            select i)
                            .Ambiguous(untilCount: 1)
            )
            .SelectMany(b => b);

            foreach (var value in parsed)
                Console.WriteLine(value);

            Console.ReadLine();
        }
    }
}
- Dave