Miru Announces OpenPGM 3

Hong Kong - April 23, 2010 - Miru, Limited, a small development studio of enterprise middleware, announces support for Solaris 10 on SPARCv9 platform for its OpenPGM messaging software, an open source low latency reliable multicast solution based on the standard for broadcasting information over an internet.  Performance for one-way messaging of roughly 75 to 99 microseconds with throughput of approximately 540 megabits per second to applications running on a single core commodity system.

The transport technology standard, known as Pragmatic General Multicast, enables private networks and the Internet to handle more traffic by sending critical business information in a more reliable, cost-effective and bandwidth-friendly manner.

The PGM reliable transport protocol communications technology, which was designed by Cisco Systems and TIBCO Software, is registered with the Internet Engineering Task Force (IETF), the Internet standards body.

PGM enabled network devices, such as Cisco, Juniper, or Nortel routers, enhance the scalability and reliability of the technology by eliminating redundant traffic when recovering lost messages.

The updated transport is supported on Linux and Solaris platforms on IA32, x86-64, SPARCv9 architectures, with other platforms and architectures added as customer needs dictate.  OpenPGM is Wire compatible with Microsoft’s PGM implementation as available in Microsoft Windows Server 2003 and Microsoft Windows XP with Microsoft Message Queuing.

About IP Multicast

In computer networking, broadcast refers to transmitting a message to every device on the network, a one-to-many paradigm similar to television or radio. Multicast is a technique to only deliver to those recipients expressing an interest in the content. A multicast source is only required to send a message once, the network infrastructure takes care of replicating to each receiver as necessary. Conventional unicast applications require the server to send copies of the same message to each recipient.

Multicast does not guarantee reliability or ordering of messages. A recipient may receive messages out of order, duplicated, or missing with no notice.

About Pragmatic General Multicast (PGM)

PGM is a reliable multicast transport protocol developed by a range of vendors including Cisco and TIBCO and described in RFC 3208.

About Miru, Limited.

Miru is development studio specialising in building high-quality, open source multicast message orientated middleware systems. Miru also offers support, training and consulting services to its customers worldwide.

Learn more: http://miru.hk .

LINUX is a trademark of Linus Torvalds. MIRU is a trademark of Miru, Limited. All other product and company names and marks mentioned in this document are property of their respective owners and are mentioned for identification purposes only.

Miru Announces Full IPv6 and Windows Support with Sub 100-Microsecond Latency

Hong Kong - February 20, 2010 - Miru, Limited, a small development studio of enterprise middleware, announces support for Internet Protocol version 6 (IPv6) and Microsoft Windows XP through Windows 7 platforms for its OpenPGM messaging software, an open source low latency reliable multicast solution based on the standard for broadcasting information over an internet.  Performance for one-way messaging of roughly 82 to 107 microseconds with throughput of approximately 270 megabits per second to applications running on a single core commodity system.

Miru has developed Windows platform support to remove the deficiencies in the native support for the PGM protocol, including IPv6 support and UDP encapsulation, and to provide a consistent cross platform interface for application development.

The transport technology standard, known as Pragmatic General Multicast, enables private networks and the Internet to handle more traffic by sending critical business information in a more reliable, cost-effective and bandwidth-friendly manner.

The PGM reliable transport protocol communications technology, which was designed by Cisco Systems and TIBCO Software, is registered with the Internet Engineering Task Force (IETF), the Internet standards body.

PGM enabled network devices, such as Cisco, Juniper, or Nortel routers, enhance the scalability and reliability of the technology by eliminating redundant traffic when recovering lost messages.

The updated transport is supported on Windows, Linux and Solaris platforms on IA32 and x86-64 architectures, with other platforms and architectures added as customer needs dictate.  OpenPGM is Wire compatible with Microsoft’s PGM implementation as available in Microsoft Windows Server 2003 and Microsoft Windows XP with Microsoft Message Queuing.

About IP Multicast

In computer networking, broadcast refers to transmitting a message to every device on the network, a one-to-many paradigm similar to television or radio. Multicast is a technique to only deliver to those recipients expressing an interest in the content. A multicast source is only required to send a message once, the network infrastructure takes care of replicating to each receiver as necessary. Conventional unicast applications require the server to send copies of the same message to each recipient.

Multicast does not guarantee reliability or ordering of messages. A recipient may receive messages out of order, duplicated, or missing with no notice.

About Pragmatic General Multicast (PGM)

PGM is a reliable multicast transport protocol developed by a range of vendors including Cisco and TIBCO and described in RFC 3208.

About Miru, Limited.

Miru is development studio specialising in building high-quality, open source multicast message orientated middleware systems. Miru also offers support, training and consulting services to its customers worldwide.

Learn more: http://miru.hk .

LINUX is a trademark of Linus Torvalds. MIRU is a trademark of Miru, Limited. All other product and company names and marks mentioned in this document are property of their respective owners and are mentioned for identification purposes only.

Miru Ships Standards Based Low Latency Open Source Messaging Software

Hong Kong - May 15, 2009 - Miru, Limited, a development studio of enterprise middleware and applications integration, today announced the immediate availability of OpenPGM, an open source low latency reliable multicast messaging software based on the standard for broadcasting information over an internet.

The transport technology standard, known as Pragmatic General Multicast, enables private networks and the Internet to handle more traffic by sending critical business information in a more reliable, cost-effective and bandwidth-friendly manner.

The PGM reliable transport protocol communications technology, which was designed by Cisco Systems and TIBCO Software, is registered with the Internet Engineering Task Force (IETF), the Internet standards body.

PGM enabled network devices, such as Cisco, Juniper, or Nortel routers, enhance the scalability and reliability of the technology by eliminating redundant traffic when recovering lost messages.

The initial general release is available for Linux and Solaris platforms on IA32 and x86-64 architectures and is wire compatible with Microsoft’s PGM implementation as available in Microsoft Windows Server 2003 and Microsoft Windows XP with Microsoft Message Queuing.

About IP Multicast

In computer networking, broadcast refers to transmitting a message to every device on the network, a one-to-many paradigm similar to television or radio. Multicast is a technique to only deliver to those recipients expressing an interest in the content. A multicast source is only required to send a message once, the network infrastructure takes care of replicating to each receiver as necessary. Conventional unicast applications require the server to send copies of the same message to each recipient.

Multicast does not guarantee reliability or ordering of messages. A recipient may receive messages out of order, duplicated, or missing with no notice.

About Pragmatic General Multicast (PGM)

PGM is a reliable multicast transport protocol developed by a range of vendors including Cisco and TIBCO and described in RFC 3208.

About Miru, Limited.

Miru is development studio specialising in building high-quality, open source multicast message orientated middleware systems. Miru also offers support, training and consulting services to its customers worldwide.
Learn more: http://miru.hk .


LINUX is a trademark of Linus Torvalds. MIRU is a trademark of Miru, Limited. All other product and company names and marks mentioned in this document are property of their respective owners and are mentioned for identification purposes only.

Flavours of Multicast

The reason why multicast is not prevalent on the Internet today is due to two main reasons, first is lack of support in network infrastructure.  Multicast is an optional part of the IPv4 protocol and so not every vendor has implemented support.  Second is filtering, as in what control is there over different parties sending data to any multicast group.  This is an issue as multicast uses a separate range of IP addresses for its communication of which is a limited number.

As an example, imagine the US President Obama’s inauguration is being being multicast live on the Internet, what happens if at the same time a radio station in New Zealand is broadcasting live news, the Hong Kong Stock Exchange is publishing stock prices, and Wembley stadium sending live match details from London?  The answer is a mess, wasted routing and link resources forwarding packets from all around to world to parties simply not interested.

This method of multicast, the default operation, is called any-source multicast (ASM), and is more suited to controlled environments such as private networks in which applications and network topology can be arranged to suit the expected usage and conflicts from other applications is not going to occur.

Source-specific multicast (SSM) was then created to limit the source of packets to a selected range of addresses.  This requires end-point router support, IGMPv3 for IPv4, and MLDv2 for IPv6, together with operating system support for the matching API and filtering without router support.

Send send send

When sending messages that a larger than one TSDU in size multiple options start to appear, some options are tied to the network layer properties for optimum transmission efficiency, some more generic to simplify application development. The following chart lists the options:

The first and second option covers the basic simplified application layer high level function, pass one or a vector of application defined message buffers. OpenPGM will then segment those buffers to the TSDU size determined from the maximum TPDU and PGM header requirements.

A traditional scatter/gather IO vector can be used with the last call, pgm_transport_sendv3(), this provides a convenient mechanism to pass an application protocol header and payload separately without copy overhead.

The sendv2() pair provides an optimised mechanism of passing PGM payload size buffers which can be directly sent on the wire with pre-pended header.

Sudoku error correction

Forward error correction (FEC) is a method of adding extra information (redundancy) to a message so that if any part is lost the data can be reconstructed without re-requesting from the sender. In a network protocol this is advantageous when either there is a significant number of receivers, e.g. internet radio, or the communications link to the sender is slow or expensive, e.g. deep space probes.

As a terse example of FEC, if a message comprised of nine numbers 1-9, and we added eight redundant numbers we end up with something like a Sudoku board:

As per the rules of Sudoku, if some numbers were missing we could determine the lost numbers from the neighbours on the same line or box.

Reed-Solomon encoding creates a graph based on a polynomial function that each point matches a byte in the data stream, x is the location in the stream, y is the value. For example, in the polynomial graph below imagine every red point being a byte of information in a transmission group. The graph can be extended to include extra data points, here marked in green. These points are extra redundant information, called parity data. As the parity points follow the same line it is possible to use these points to re-construct the original graph polynomial function. Once this function is calculated any missing real data points can be recovered by substituting the x location values.

The benefit over convential selective "Automatic Repeat reQuest" (ARQ), is that one parity point can recover any one lost original data point. The disadvantage is the extra time to perform the calculations, however in hardware systems these calculations can be implemented directly in hardware using a slightly different form called BCH Code.

Both forms of code are popular in software projects, notable examples include Luigi Rizzo's RMDP, Peter Brian Clements PAR Parity Archives, and Phil Karn's DSP and FEC library (e.g. Linux software modems). However the results are in different forms, Vandermonde calculations produce vector space coefficients, and BCH's Linear Shift Feedback Register produces polynomial space coefficients. Microsoft's PGM implementation uses Rizzo's implementation, and so for initial compatibility OpenPGM will use a Vandermonde matrix calculation.

Network system testing

Testing is always helpful in development, large projects often undergo testing at different levels: unit testing, integration testing, and performance testing. With a multicast network protocol none of these cover actual testing of the protocol between hosts, so we create a new method: network system testing. We want to test the OpenPGM stack and the API it provides to the application developer as pictured below on the top right.

Some tests need an external source to drive functionality in the stack, the Simulator is used for this task. In order to verify the packets sent out by the stack are correct with have the Monitor.

In order to build an extensive set of tests that can be reliably re-run we want to use automated testing. This means some form of scripting of all three systems and synchronisation between how each is run. The Tester host runs a script that remotely controls and receives feedback from each of the three test systems. All communication is via stdin and stdout, including the monitor with is a glorified version of tcpdump but shows PGM packets in JSON form.

To make everything platform agnostic and to ease development all scripts are in Perl, modules can be used to SSH into remote hosts, perform high resolution timing, process JSON representation of PGM packets, etc.