We believe performance is one of the distinguishing features of Fix8. Here we will present our comparison test between Fix8 and Quickfix and also our current encode and decode performance stats along with a description of our test environment and test methodology.






1. Our test environment

Our test platform has the following specifications:

Fedora release 18 (Spherical Cow)
Linux Version 3.9.2-200.fc18.x86_64 GNU/Linux
Compiled #1 SMP Mon May 13 13:59:47 UTC 2013
Thirty-two (16x2 HT) Intel(R) Xeon(R) CPU E5-2690 @ 2.90GHz
20480 KB Cache; 128GB RAM
185691 BogoMIPS Total

And we are building with:

gcc version 4.7.2 20121109 (Red Hat 4.7.2-8) (GCC)

We are testing with the following library versions:

poco version 1.4.6
gperftools 2.0 (for tcmalloc)
quickfix version 1.13.3
tbb version 4.2 (tbb42_20131003oss)




2. Comparison With Quickfix

This is perhaps the most commonly asked question about Fix8. We thought we'd present how we compared the two frameworks, how we tested Fix8 and QuickFix and demonstrate that Fix8 outperforms.

Our test applications

We have two almost identical applications - one Fix8 and one QuickFix - that generate 100000 NewOrderSingle(D) messages. The encode and decode times are accumulated for each test and reported at test completion.

Note that we used tcmalloc for this test. See the FAQ for details.

#include "quickfix/Field.h"
#include "quickfix/Values.h"
#include "quickfix/Message.h"
#include "tbb/tick_count.h"

//-------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
    FIX::Message msg;
    msg.getHeader().setField( FIX::BeginString( "FIX.4.4" ) );
    msg.getHeader().setField( FIX::MsgType( FIX::MsgType_NewOrderSingle ) );
    msg.getHeader().setField( FIX::MsgSeqNum(78));
    msg.getHeader().setField(FIX::SenderCompID("A12345B"));
    msg.getHeader().setField(FIX::SenderSubID("2DEFGH4"));
    msg.getHeader().setField(FIX::SendingTime(FIX::UtcTimeStamp()));
    msg.getHeader().setField(FIX::TargetCompID("COMPARO"));
    msg.getHeader().setField(FIX::TargetSubID("G"));
    msg.getHeader().setField(FIX::SenderLocationID("AU,SY"));

    msg.setField( FIX::Account( "01234567") );
    msg.setField( FIX::ClOrdID( "4" ) );
    msg.setField( FIX::OrderQty( 50 ) );
    msg.setField( FIX::OrdType( FIX::OrdType_LIMIT) );
    msg.setField( FIX::Price( 400.5) );
    msg.setField( FIX::HandlInst( '1' ) );
    msg.setField( FIX::Symbol( "OC") );
    msg.setField( FIX::Text( "NIGEL") );
    msg.setField( FIX::Side( FIX::Side_BUY ) );
    msg.setField( FIX::SecurityDesc( "AOZ3 C02000") );
    msg.setField( FIX::TimeInForce( FIX::TimeInForce_DAY ) );
    msg.setField( FIX::TransactTime() );
    msg.setField( FIX::SecurityType( FIX::SecurityType_OPTION ) );

    std::string output;
    double tt = 0;
    for(int i = 0 ; i < 100000; ++i)
    {
        FIX::Message msg;
        msg.getHeader().setField( FIX::BeginString( "FIX.4.4" ) );
        msg.getHeader().setField( FIX::MsgType( FIX::MsgType_NewOrderSingle ) );
        msg.getHeader().setField( FIX::MsgSeqNum(78));
        msg.getHeader().setField(FIX::SenderCompID("A12345B"));
        msg.getHeader().setField(FIX::SenderSubID("2DEFGH4"));
        msg.getHeader().setField(FIX::SendingTime(FIX::UtcTimeStamp()));
        msg.getHeader().setField(FIX::TargetCompID("COMPARO"));
        msg.getHeader().setField(FIX::TargetSubID("G"));
        msg.getHeader().setField(FIX::SenderLocationID("AU,SY"));

        msg.setField( FIX::Account( "01234567") );
        msg.setField( FIX::ClOrdID( "4" ) );
        msg.setField( FIX::OrderQty( 50 ) );
        msg.setField( FIX::OrdType( FIX::OrdType_LIMIT) );
        msg.setField( FIX::Price( 400.5) );
        msg.setField( FIX::HandlInst( '1' ) );
        msg.setField( FIX::Symbol( "OC") );
        msg.setField( FIX::Text( "NIGEL") );
        msg.setField( FIX::Side( FIX::Side_BUY ) );
        msg.setField( FIX::SecurityDesc( "AOZ3 C02000") );
        msg.setField( FIX::TimeInForce( FIX::TimeInForce_DAY ) );
        msg.setField( FIX::TransactTime() );
        msg.setField( FIX::SecurityType( FIX::SecurityType_OPTION ) );

        tbb::tick_count start = tbb::tick_count::now();
        msg.toString(output);
        tbb::tick_count end = tbb::tick_count::now();

        tt += (end-start).seconds();
    }
    std::cout << "to string - " << tt << std::endl;

    FIX::DataDictionary dict("/usr/local/share/quickfix/FIX44.xml");
    tbb::tick_count start = tbb::tick_count::now();
    for(int i = 0 ; i < 100000; ++i)
    {
        FIX::Message * tmp = new FIX::Message(output, dict);
        delete tmp;
    }
    tbb::tick_count end = tbb::tick_count::now();
    std::cout << "from string - " << (end-start).seconds() << std::endl;
    return 0;
}
// f8 headers
#include "f8includes.hpp"
#include "message.hpp"
#include "tbb/tick_count.h"
#include "COMPAROFIX_types.hpp"
#include "COMPAROFIX_router.hpp"
#include "COMPAROFIX_classes.hpp"

using namespace FIX8::COMPAROFIX;
using namespace FIX8;

//-------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
    NewOrderSingle *nos(new NewOrderSingle);
    *nos->Header() << new msg_seq_num(78)
                   << new sender_comp_id("A12345B")
                   << new SenderSubID("2DEFGH4")
                   << new sending_time
                   << new target_comp_id("COMPARO")
                   << new TargetSubID("G")
                   << new SenderLocationID("AU,SY");

    *nos << new TransactTime
         << new Account("01234567")
         << new OrderQty(50)
         << new Price(400.5)
         << new ClOrdID("4")
         << new HandlInst(HandlInst_AUTOMATED_EXECUTION_ORDER_PRIVATE)
         << new OrdType(OrdType_LIMIT)
         << new Side(Side_BUY)
         << new Symbol("OC")
         << new Text("NIGEL")
         << new TimeInForce(TimeInForce_DAY)
         << new SecurityDesc("AOZ3 C02000")
         << new SecurityType(SecurityType_OPTION);

    std::string output, tmp;
    nos->encode(output);
    double tt = 0;

    // quickfix doesn't encode chksum till send, so we can suppress here
    Message::set_no_chksum_flag(true);

    for(int i = 0 ; i < 100000; ++i)
    {
        NewOrderSingle *nos(new NewOrderSingle);
        *nos->Header() << new msg_seq_num(78)
                       << new sender_comp_id("A12345B")
                       << new SenderSubID("2DEFGH4")
                       << new sending_time
                       << new target_comp_id("COMPARO")
                       << new TargetSubID("G")
                       << new SenderLocationID("AU,SY");

        *nos  << new TransactTime
              << new Account("01234567")
              << new OrderQty(50)
              << new Price(400.5)
              << new ClOrdID("4")
              << new HandlInst(HandlInst_AUTOMATED_EXECUTION_ORDER_PRIVATE)
              << new OrdType(OrdType_LIMIT)
              << new Side(Side_BUY)
              << new Symbol("OC")
              << new Text("NIGEL")
              << new TimeInForce(TimeInForce_DAY)
              << new SecurityDesc("AOZ3 C02000")
              << new SecurityType(SecurityType_OPTION);

        tbb::tick_count start = tbb::tick_count::now();
        nos->encode(tmp);
        tbb::tick_count end = tbb::tick_count::now();
        tt += (end-start).seconds();

        delete nos;
    }
    std::cout  << "to string- " << tt << std::endl;

    FIX8::Message * msg = NULL;
    tbb::tick_count start = tbb::tick_count::now();
    for(int i = 0 ; i < 100000; ++i)
    {
        msg = FIX8::Message::factory(FIX8::COMPAROFIX::ctx(), output);
        delete msg;
    }

    tbb::tick_count end = tbb::tick_count::now();
    std::cout << "from string - " <<  (end-start).seconds() << std::endl;
    return 0;
}

The results

We ran the test 10 times with each version and averaged the result.

Framework Operation Msgs processed Time secs Msgs/sec Average µs/msg % Reduction Thoughput
Quickfix encode 100000 0.4560319 219282 4.57 - -
decode 100000 1.364743 73274 13.65 - -
Fix8 encode 100000 0.1429087 699747

1.43

68.7

3.2x

decode 100000 0.4495174 222461

4.50

67.0

3.0x


Conclusion

For the same message, Fix8 encodes 3.2 times faster and decodes 3.0 times faster for an average improvement of 3.0 times over Quickfix. In other words, reduces encode latency by 69% and reduces decode latency by 67%.





3. Encode/Decode Performance Test

This section describes the supplied test client/server that you can build and test to compare your performance wth our published results.

You will need at least 16GB of RAM to run the full test. If you have 8GB, limit the number of preloaded messages to 200000. For 4GB you will need to limit the number of messages to 100000.


The message set

The test client sends a simple NewOrderSingle(D) message for a Fill or Kill buy order with a random quantity bwtween 1 and 10000 and a random price between 1.0 and 500.0. Each order will have a unique ClOrderID. The following is a typical order.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 146
   MsgType (35): ORDER_SINGLE (D)
   SenderCompID (49): DLD_TEX
   TargetCompID (56): TEX_DLD
   MsgSeqNum (34): 499650
   SendingTime (52): 20121227-11:20:43
NewOrderSingle ("D")
   ClOrdID (11): ord509647-500000
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
   Symbol (55): BHP
   Side (54): BUY (1)
   TransactTime (60): 20121227-11:20:14
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.075
   TimeInForce (59): FILL_OR_KILL (4)
trailer ("trailer")
   CheckSum (10):

When the test server receives a NewOrderSingle message it sends either an ExecutionReport(8) with order reject, order cancel or order confirmation (randomly selected). The following is a typical order confirmation.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 224
   MsgType (35): EXECUTION_REPORT (8)
   SenderCompID (49): TEX_DLD
   TargetCompID (56): DLD_TEX
   MsgSeqNum (34): 1962926
   SendingTime (52): 20121227-11:22:07
ExecutionReport ("8")
   OrderID (37): ord499647
   ClOrdID (11): ord509647-500000
   ExecID (17): ord499647
   ExecTransType (20): NEW (0)
   ExecType (150): NEW (0)
   OrdStatus (39): NEW (0)
   Symbol (55): BHP
   Side (54): BUY (1)
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.07
   TimeInForce (59): FILL_OR_KILL (4)
   LastCapacity (29): 5
   LeavesQty (151): 57
   CumQty (14): 0
   AvgPx (6): 0
   TransactTime (60): 20121227-11:20:14
   ReportToExch (113): YES (Y)
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
   CheckSum (10):

Followed by one or more fills.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 217
   MsgType (35): EXECUTION_REPORT (8)
   SenderCompID (49): TEX_DLD
   TargetCompID (56): DLD_TEX
   MsgSeqNum (34): 1962929
   SendingTime (52): 20121227-11:22:07
ExecutionReport ("8")
   OrderID (37): ord499647
   ClOrdID (11): ord509647-500000
   ExecID (17): exec1463279
   ExecTransType (20): NEW (0)
   ExecType (150): NEW (0)
   OrdStatus (39): FILLED (2)
   Symbol (55): BHP
   Side (54): BUY (1)
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.07
   TimeInForce (59): FILL_OR_KILL (4)
   LeavesQty (151): 0
   CumQty (14): 57
   AvgPx (6): 298.07
   TransactTime (60): 20121227-11:20:14
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
   CheckSum (10):

The test client/server

Note that we used tcmalloc and tbb for this test. See the FAQ for details.

Before you begin, set your compiler optimisation to -O3

% export CXXFLAGS=-O3
You then need to pass the codec timing switch --enable-codectiming and the no fill message metadata switch
--enable-fillmetadata=no to configure and cleanly build Fix8.
% ./configure --prefix=[your target directory] --enable-codectiming --with-mpmc=tbb --enable-doxygen=no --enable-fillmetadata=no
% make clean; make install

If you have installed Fix8 from a release rpm, the supplied hftest will not be configured for this test. Please rebuild it following these instructions.

For our test, we will simulate an HF client preloading and sending 500000 orders to a server. We will measure the encode and decode timings for both the client and the server. We are also going to use the coroutine process mode, which will minimise the effects of jitter. The test client and server hftest are built by default when you build Fix8. Also by default, the xml configuration files hf_server.xml, hf_client.xml, hf_client_include.xml needed are supplied already preconfigured.

Do not use the configure switches shown above when building the other supplied test applications.

To run the server, execute the following command (assuming the xml config files are in the current directory):

% ./hftest -sol server
In another terminal, run the client:
% ./hftest -l client -p 500000 -u 50000 -c coro_hf_client.xml

We are telling the client you will be preloading 500000 messages. If you find the client disconnects during the test, restart the client in reliable mode:

% ./hftest -rl client -p 500000 -u 50000 -c coro_hf_client.xml
0 NewOrderSingle msgs currently preloaded.
loading...
0000001 A 2013-04-19 17:19:50.512934156 Starting session
0000002 A 2013-04-19 17:19:50.513103903 Trying to connect to: 127.0.0.1:11002 (1)
0000003 A 2013-04-19 17:19:50.513403209 Connection successful
0000004 A 2013-04-19 17:19:50.513421223 Session connected
0000005 B 2013-04-19 17:19:50.514467212 Client setting heartbeat interval to 10
0000006 B 2013-04-19 17:19:50.514470236 Heartbeat interval is 10
500000 NewOrderSingle msgs preloaded.

When the client has connected (there are no on screen messages), press ? to see the help menu...

Key     Command
===     =======
?       Help
N       Send n NewOrderSingle msgs
a       Send all Preloaded NewOrderSingle msgs
b       Batch preload and send n NewOrderSingle msgs
l       Logout
n       Send a NewOrderSingle msg
p       Preload n NewOrderSingle msgs
x       Exit

Now press 'a' to send the preloaded orders to the server:

500000 NewOrderSingle msgs sent
0 NewOrderSingle msgs remaining.
1967000 ExecutionReport msgs received

When all the execution reports have been received (in the client window), press 'l' to logout and exit. Wait a few moments for the logging buffer to empty. When the client has disconnected the server will also flush its logging buffer so wait a bit longer for this as well. Occasionally you may get a core dump on exit - you can ignore this for now. Your server should have also now exited (we passed the o option).

You can use the supplied printer application hfprint to examine the protocol log output (you will need to enable the file protocol target in the xml configuration).


Preparing the results

If all has gone to plan you should have two log files, server and client. Examine the contents of these two files to see the results.

% cat client server|egrep "Encode|Decode"
0000004 A 2013-11-05 13:51:25.203329321 client Encode: 1.027829627 secs,   500003 msgs, 0.000002056 secs/msg, 486464.86 msgs/sec
0000005 A 2013-11-05 13:51:25.203390025 client Decode: 6.196271045 secs,  1960918 msgs, 0.000003160 secs/msg, 316467.43 msgs/sec
0000006 A 2013-11-05 13:51:24.224960404 server Encode: 4.069362531 secs,  1960918 msgs, 0.000002075 secs/msg, 481873.51 msgs/sec
0000007 A 2013-11-05 13:51:24.225002281 server Decode: 1.109719646 secs,   500003 msgs, 0.000002219 secs/msg, 450566.95 msgs/sec

The results

Here are the tabulated results.


Mode Operation Msgs processed Time secs Msgs/sec Average µs/msg
client encode 500003 1.03 486465 2.06
decode 1960918 6.20 316467 3.16
server encode 1960918 4.07 481874 2.08
decode 500003 1.11 450567 2.22

Averaged across client and server...

Operation Average µs/msg
encode 2.07
decode 2.69

Conclusion

Most users will be interested in client encode and decode. On our test hardware, Fix8 encodes a NewOrderSingle(D) in 2.07 µs and decodes an ExecutionReport(8) in 2.69 µs when compiled with gcc. It can also be seen that when averaged across client and server, encode performance is better than decode.




  • Home
  • Fix8MT
  • How It Works
  • Downloads
  • FAQ
  • News
  • Wiki
  • Support
  • Partners
  • Repository
  • API Documentation
  • About Us
  • Register


  • Sponsorship helps our project.