Our test platform has the following specifications:
Fedora release 18 (Spherical Cow) Linux Version 3.8.9-200.fc18.x86_64 GNU/Linux Compiled #1 SMP Fri Apr 26 12:50:07 UTC 2013 Thirty-two (16x2 HT) Intel(R) Xeon(R) CPU E5-2690 @ 2.90GHz 20480 KB Cache; 128GB RAM 185691 BogoMIPS Total
And we are building with:
gcc version 4.7.2 20121109 (Red Hat 4.7.2-8) (GCC)
We are testing with the following library versions:
poco version 1.4.6 gperftools 2.0 (for tcmalloc) quickfix version 1.13.3 tbb version 4.1 (tbb41_20130116oss)
This is perhaps the most commonly asked question about Fix8. We thought we'd present how we compared the two frameworks, how we tested Fix8 and QuickFix and demonstrate that Fix8 outperforms.
We have two almost identical applications - one Fix8 and one QuickFix - that generate 100000 NewOrderSingle(D) messages. The encode and
decode times are accumulated for each test and reported at test completion.
#include "quickfix/Field.h"
#include "quickfix/Values.h"
#include "quickfix/Message.h"
#include "tbb/tick_count.h"
//-------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
FIX::Message msg;
msg.getHeader().setField( FIX::BeginString( "FIX.4.4" ) );
msg.getHeader().setField( FIX::MsgType( FIX::MsgType_NewOrderSingle ) );
msg.getHeader().setField( FIX::MsgSeqNum(78));
msg.getHeader().setField(FIX::SenderCompID("A12345B"));
msg.getHeader().setField(FIX::SenderSubID("2DEFGH4"));
msg.getHeader().setField(FIX::SendingTime(FIX::UtcTimeStamp()));
msg.getHeader().setField(FIX::TargetCompID("COMPARO"));
msg.getHeader().setField(FIX::TargetSubID("G"));
msg.getHeader().setField(FIX::SenderLocationID("AU,SY"));
msg.setField( FIX::Account( "01234567") );
msg.setField( FIX::ClOrdID( "4" ) );
msg.setField( FIX::OrderQty( 50 ) );
msg.setField( FIX::OrdType( FIX::OrdType_LIMIT) );
msg.setField( FIX::Price( 400.5) );
msg.setField( FIX::HandlInst( '1' ) );
msg.setField( FIX::Symbol( "OC") );
msg.setField( FIX::Text( "NIGEL") );
msg.setField( FIX::Side( FIX::Side_BUY ) );
msg.setField( FIX::SecurityDesc( "AOZ3 C02000") );
msg.setField( FIX::TimeInForce( FIX::TimeInForce_DAY ) );
msg.setField( FIX::TransactTime() );
msg.setField( FIX::SecurityType( FIX::SecurityType_OPTION ) );
std::string output;
double tt = 0;
for(int i = 0 ; i < 100000; ++i)
{
FIX::Message msg;
msg.getHeader().setField( FIX::BeginString( "FIX.4.4" ) );
msg.getHeader().setField( FIX::MsgType( FIX::MsgType_NewOrderSingle ) );
msg.getHeader().setField( FIX::MsgSeqNum(78));
msg.getHeader().setField(FIX::SenderCompID("A12345B"));
msg.getHeader().setField(FIX::SenderSubID("2DEFGH4"));
msg.getHeader().setField(FIX::SendingTime(FIX::UtcTimeStamp()));
msg.getHeader().setField(FIX::TargetCompID("COMPARO"));
msg.getHeader().setField(FIX::TargetSubID("G"));
msg.getHeader().setField(FIX::SenderLocationID("AU,SY"));
msg.setField( FIX::Account( "01234567") );
msg.setField( FIX::ClOrdID( "4" ) );
msg.setField( FIX::OrderQty( 50 ) );
msg.setField( FIX::OrdType( FIX::OrdType_LIMIT) );
msg.setField( FIX::Price( 400.5) );
msg.setField( FIX::HandlInst( '1' ) );
msg.setField( FIX::Symbol( "OC") );
msg.setField( FIX::Text( "NIGEL") );
msg.setField( FIX::Side( FIX::Side_BUY ) );
msg.setField( FIX::SecurityDesc( "AOZ3 C02000") );
msg.setField( FIX::TimeInForce( FIX::TimeInForce_DAY ) );
msg.setField( FIX::TransactTime() );
msg.setField( FIX::SecurityType( FIX::SecurityType_OPTION ) );
tbb::tick_count start = tbb::tick_count::now();
msg.toString(output);
tbb::tick_count end = tbb::tick_count::now();
tt += (end-start).seconds();
}
std::cout << "to string - " << tt << std::endl;
FIX::DataDictionary dict("/usr/local/share/quickfix/FIX44.xml");
tbb::tick_count start = tbb::tick_count::now();
for(int i = 0 ; i < 100000; ++i)
{
FIX::Message * tmp = new FIX::Message(output, dict);
delete tmp;
}
tbb::tick_count end = tbb::tick_count::now();
std::cout << "from string - " << (end-start).seconds() << std::endl;
return 0;
}
// f8 headers
#include "f8includes.hpp"
#include "message.hpp"
#include "tbb/tick_count.h"
#include "COMPAROFIX_types.hpp"
#include "COMPAROFIX_router.hpp"
#include "COMPAROFIX_classes.hpp"
using namespace FIX8::COMPAROFIX;
using namespace FIX8;
//-------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
NewOrderSingle *nos(new NewOrderSingle);
*nos->Header() << new msg_seq_num(78)
<< new sender_comp_id("A12345B")
<< new SenderSubID("2DEFGH4")
<< new sending_time
<< new target_comp_id("COMPARO")
<< new TargetSubID("G")
<< new SenderLocationID("AU,SY");
*nos << new TransactTime
<< new Account("01234567")
<< new OrderQty(50)
<< new Price(400.5)
<< new ClOrdID("4")
<< new HandlInst(HandlInst_AUTOMATED_EXECUTION_ORDER_PRIVATE)
<< new OrdType(OrdType_LIMIT)
<< new Side(Side_BUY)
<< new Symbol("OC")
<< new Text("NIGEL")
<< new TimeInForce(TimeInForce_DAY)
<< new SecurityDesc("AOZ3 C02000")
<< new SecurityType(SecurityType_OPTION);
std::string output, tmp;
nos->encode(output);
double tt = 0;
for(int i = 0 ; i < 100000; ++i)
{
NewOrderSingle *nos(new NewOrderSingle);
*nos->Header() << new msg_seq_num(78)
<< new sender_comp_id("A12345B")
<< new SenderSubID("2DEFGH4")
<< new sending_time
<< new target_comp_id("COMPARO")
<< new TargetSubID("G")
<< new SenderLocationID("AU,SY");
*nos << new TransactTime
<< new Account("01234567")
<< new OrderQty(50)
<< new Price(400.5)
<< new ClOrdID("4")
<< new HandlInst(HandlInst_AUTOMATED_EXECUTION_ORDER_PRIVATE)
<< new OrdType(OrdType_LIMIT)
<< new Side(Side_BUY)
<< new Symbol("OC")
<< new Text("NIGEL")
<< new TimeInForce(TimeInForce_DAY)
<< new SecurityDesc("AOZ3 C02000")
<< new SecurityType(SecurityType_OPTION);
tbb::tick_count start = tbb::tick_count::now();
nos->encode(tmp);
tbb::tick_count end = tbb::tick_count::now();
tt += (end-start).seconds();
delete nos;
}
std::cout << "to string- " << tt << std::endl;
FIX8::Message * msg = NULL;
tbb::tick_count start = tbb::tick_count::now();
for(int i = 0 ; i < 100000; ++i)
{
msg = FIX8::Message::factory(FIX8::COMPAROFIX::ctx, output);
delete msg;
}
tbb::tick_count end = tbb::tick_count::now();
std::cout << "from string - " << (end-start).seconds() << std::endl;
return 0;
}
We ran the test 10 times with each version and averaged the result.
| Framework | Operation | Msgs processed | Time secs | Msgs/sec | Average µs/msg | % Improvement | Thoughput |
|---|---|---|---|---|---|---|---|
| Quickfix | encode | 100000 | 0.529 | 188695.409 | 5.29 | - | - |
| decode | 100000 | 1.527 | 65499.295 | 15.27 | - | - | |
| Fix8 | encode | 100000 | 0.192 | 520827.366 | 1.92 | 63.8 |
2.8x |
| decode | 100000 | 0.975 | 102578.748 | 9.75 | 36.2 |
1.6x |
For the same message, Fix8 encodes 2.8 times faster and decodes 1.6 times faster for an average improvement of 2 times over Quickfix. In other words, reduces encode latency by 64% and reduces decode latency by 36%.
This section describes the supplied test client/server that you can build and test to compare your performance wth our published results.
You will need at least 16GB of RAM to run the full test. If you have 8GB, limit the number of preloaded messages to 200000. For 4GB you will need to limit the number of messages to 100000.
The test client sends a simple NewOrderSingle(D) message for a Fill or Kill buy order with a random quantity bwtween 1 and 10000
and a random price between 1.0 and 500.0. Each order will have a unique ClOrderID. The following is a typical order.
header ("header")
BeginString (8): FIX.4.2
BodyLength (9): 146
MsgType (35): ORDER_SINGLE (D)
SenderCompID (49): DLD_TEX
TargetCompID (56): TEX_DLD
MsgSeqNum (34): 499650
SendingTime (52): 20121227-11:20:43
NewOrderSingle ("D")
ClOrdID (11): ord509647-500000
HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
Symbol (55): BHP
Side (54): BUY (1)
TransactTime (60): 20121227-11:20:14
OrderQty (38): 57
OrdType (40): LIMIT (2)
Price (44): 298.075
TimeInForce (59): FILL_OR_KILL (4)
trailer ("trailer")
CheckSum (10):
When the test server receives a NewOrderSingle message it sends either an ExecutionReport(8) with order reject,
order cancel or order confirmation (randomly selected). The following is a typical order confirmation.
header ("header")
BeginString (8): FIX.4.2
BodyLength (9): 224
MsgType (35): EXECUTION_REPORT (8)
SenderCompID (49): TEX_DLD
TargetCompID (56): DLD_TEX
MsgSeqNum (34): 1962926
SendingTime (52): 20121227-11:22:07
ExecutionReport ("8")
OrderID (37): ord499647
ClOrdID (11): ord509647-500000
ExecID (17): ord499647
ExecTransType (20): NEW (0)
ExecType (150): NEW (0)
OrdStatus (39): NEW (0)
Symbol (55): BHP
Side (54): BUY (1)
OrderQty (38): 57
OrdType (40): LIMIT (2)
Price (44): 298.07
TimeInForce (59): FILL_OR_KILL (4)
LastCapacity (29): 5
LeavesQty (151): 57
CumQty (14): 0
AvgPx (6): 0
TransactTime (60): 20121227-11:20:14
ReportToExch (113): YES (Y)
HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
CheckSum (10):
Followed by one or more fills.
header ("header")
BeginString (8): FIX.4.2
BodyLength (9): 217
MsgType (35): EXECUTION_REPORT (8)
SenderCompID (49): TEX_DLD
TargetCompID (56): DLD_TEX
MsgSeqNum (34): 1962929
SendingTime (52): 20121227-11:22:07
ExecutionReport ("8")
OrderID (37): ord499647
ClOrdID (11): ord509647-500000
ExecID (17): exec1463279
ExecTransType (20): NEW (0)
ExecType (150): NEW (0)
OrdStatus (39): FILLED (2)
Symbol (55): BHP
Side (54): BUY (1)
OrderQty (38): 57
OrdType (40): LIMIT (2)
Price (44): 298.07
TimeInForce (59): FILL_OR_KILL (4)
LeavesQty (151): 0
CumQty (14): 57
AvgPx (6): 298.07
TransactTime (60): 20121227-11:20:14
HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
CheckSum (10):
Before you begin, set your compiler optimisation to -O3
% export CXXFLAGS=-O3You then need to pass the codec timing switch
--enable-codectiming and the no fill message metadata
switch--enable-fillmetadata=no to configure and cleanly build Fix8.
% ./configure --prefix=[your target directory] --enable-codectiming --with-mpmc=tbb --enable-doxygen=no --enable-fillmetadata=no % make clean; make install
If you have installed Fix8 from a release rpm, the supplied hftest will not be configured for this test. Please rebuild it following these instructions.
For our test, we will simulate an HF client preloading and sending 500000 orders to a server. We will measure the encode and decode timings for both the client and the server. The test client and serverhftest are built by default when you build Fix8.
Also by default, the xml configuration files hf_server.xml, hf_client.xml, hf_client_include.xml needed are supplied
already preconfigured.
Do not use the configure switches shown above when building the other supplied test applications.
To run the server, execute the following command (assuming the xml config files are in the current directory):
% ./hftest -sl serverIn another terminal, run the client:
% ./hftest -l client -p 500000
We are telling the client you will be preloading 500000 messages. If you find the client disconnects during the test, restart the client in reliable mode:
% ./hftest -rl client -p 500000 0 NewOrderSingle msgs currently preloaded. loading... 0000001 A 2013-04-19 17:19:50.512934156 Starting session 0000002 A 2013-04-19 17:19:50.513103903 Trying to connect to: 127.0.0.1:11002 (1) 0000003 A 2013-04-19 17:19:50.513403209 Connection successful 0000004 A 2013-04-19 17:19:50.513421223 Session connected 0000005 B 2013-04-19 17:19:50.514467212 Client setting heartbeat interval to 10 0000006 B 2013-04-19 17:19:50.514470236 Heartbeat interval is 10 500000 NewOrderSingle msgs preloaded.
When the client has connected (there are no on screen messages), press ? to see the help menu...
Key Command === ======= ? Help N Send n NewOrderSingle msgs a Send all Preloaded NewOrderSingle msgs b Batch preload and send n NewOrderSingle msgs l Logout n Send a NewOrderSingle msg p Preload n NewOrderSingle msgs x Exit
Now press 'a' to send the preloaded orders to the server:
500000 NewOrderSingle msgs sent 0 NewOrderSingle msgs remaining. 1967000 ExecutionReport msgs received
When all the execution reports have been received (in the client window), press 'l' to logout and exit. Wait a few moments for the
logging buffer to empty. When the client has disconnected the server will also flush its logging buffer so wait a bit longer for this as well.
Occasionally you may get a core dump on exit - you can ignore this for now. Switch to your server and press ^C to terminate the server.
You can use the supplied printer application hfprint to examine the protocol log output (you will need to enable the file protocol target in the xml configuration).
If all has gone to plan you should have two log files, server and client. Examine the contents of these
two files to see the results.
% cat client server|egrep "Encode|Decode" 0000003 A 2013-05-03 20:15:00.855875931 Encode: 1.372111365 secs, 500005 msgs, 0.000002744 secs/msg, 364405.55 msgs/sec 0000004 A 2013-05-03 20:15:00.855901905 Decode: 13.087327673 secs, 1963246 msgs, 0.000006666 secs/msg, 150011.22 msgs/sec 0000005 A 2013-05-03 20:14:57.462742499 Encode: 5.719571982 secs, 1967623 msgs, 0.000002907 secs/msg, 344015.78 msgs/sec 0000006 A 2013-05-03 20:14:57.909708719 Decode: 2.238811756 secs, 500004 msgs, 0.000004478 secs/msg, 223334.54 msgs/sec
Here are the tabulated results.
| Mode | Operation | Msgs processed | Time secs | Msgs/sec | Average µs/msg |
|---|---|---|---|---|---|
| client | encode | 500005 | 1.37 | 364405 | 2.74 |
| decode | 1963246 | 13.09 | 150011 | 6.67 | |
| server | encode | 1963246 | 5.72 | 344015 | 2.91 |
| decode | 500005 | 2.24 | 223334 | 4.48 |
Averaged across client and server...
| Operation | Average µs/msg |
|---|---|
| encode | 2.83 |
| decode | 5.57 |
Most users will be interested in client encode and decode. On our test hardware, Fix8 encodes a NewOrderSingle(D) in
2.74 µs and decodes an ExecutionReport(8) in 6.67 µs when compiled with gcc. It can also be seen that when averaged across client and server, encode performance is better than decode.