{"id":398,"date":"2017-11-05T17:53:11","date_gmt":"2017-11-05T17:53:11","guid":{"rendered":"http:\/\/evolvedmicrobe.com\/blogs\/?p=398"},"modified":"2017-11-05T20:34:35","modified_gmt":"2017-11-05T20:34:35","slug":"net-bio-is-significantly-faster-on-net-core-2-0","status":"publish","type":"post","link":"http:\/\/evolvedmicrobe.com\/blogs\/?p=398","title":{"rendered":".NET Bio is Significantly Faster on .Net Core 2.0"},"content":{"rendered":"Summary: With the release of .NET Core 2.0, .NET Bio is able to run significantly faster (~2X) on Mac OSX due to better compilation and memory mangement.<\/strong>\r\n

\r\nThe .NET Bio<\/a>\u00a0library contains libraries for genomic data processing tasks like parsing, alignment, etc. that are too computationally intense to be\u00a0undertaken with interpreted languages like Python or R, but can be efficiently performed in code that is typed and compiled, like Java and C#, without suffering the productivity burden of coding in C\/C++.\r\n\r\n

\r\n\r\nHistorically, .NET Bio ran on two different frameworks.\u00a0 On windows one could leverage all the substantial engineering Microsoft invested in their CLR<\/a>\u00a0to create fast and stable programs.\u00a0 On Mac or Linux, one could use the independently implemented Mono Framework<\/a>, which was a\u00a0solid effort by a significantly less resourced team that performed well, but never really came close to the quality of the Microsoft implementation.\u00a0 In practice, as bioinformaticians are weened on the Linux\/GCC toolchain and infrastructure, Mono was the frequently used option.\u00a0 This led to performance and reliability issues.\r\n

\r\nAll of this changed when Microsoft decided to open source .NET<\/a>\u00a0, releasing a version of their framework for Mac, Linux and Windows called .NET Core<\/a>.\u00a0 This was a watershed moment, as it allows open source writers to take advantage of all the advanced memory management and compilation techniques Microsoft developed.\u00a0 Better Garbage collection, advanced pre-compilation and vector instructions are all allowing higher level languages like C# to perform nearly as well as aggressively optimized C\/C++ for many practical applications.\r\n

\r\nTo test out the\u00a0new possibilites, I compared the time it takes to parse all the reads in a 1.2 GB BAM<\/strong>\u00a0file using either the old Mono runtime or the new dotnet core runtime on my MacBook computer.\u00a0 The test code was simple counting of reads after they were deserialized from the compressed format and converted into objects.\u00a0\u00a0<\/strong>\r\n

var p = new BAMParser();\r\nint count = 0;\r\nforeach (var read in p.Parse(fname))\r\n{ count += 1;}\r\nConsole.WriteLine(count);\r\n<\/pre>\r\nAnd the test case was timed on each platform as follows, the times were taken in triplicate and random order and the results are shown below.\u00a0 I also benchmarked an equivalent task using the industry standard samtools<\/strong> available from htslib<\/code><\/a>, a C library for the same task that uses tons of bit operations, and is about as performant as one can get 1)<\/sup>This is for methods that fully parse the BAM read, PySAM, which lazily decodes the data<\/a>, can finish a simple counting task in about 15 seconds, but this isn’t representative of a workflow which would require full decoding.<\/span>