We believe performance is one of the distinguishing features of Fix8. Here we will present our current encode and decode performance stats along with a description of our test environment and test methodology.

1. Our test environment

Our test platform has the following specifications:

Linux 4.19.69-1-MANJARO #1 SMP PREEMPT Thu Aug 29 08:51:46 UTC 2019 x86_64 GNU/Linux
Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
16GB RAM

And we are building with:

gcc version 9.1.0 (GCC)

We are testing with the following library versions:

poco version 1.8.0
tbb version 2018_U1

2. Encode/Decode Performance Test

This section describes the supplied test client/server that you can build and test to compare your performance wth our published results.

You will need at least 16GB of RAM to run the full test. If you have 8GB, limit the number of preloaded messages to 200000. For 4GB you will need to limit the number of messages to 100000.

The message set

The test client sends a simple NewOrderSingle(D) message for a Fill or Kill buy order with a random quantity bwtween 1 and 10000 and a random price between 1.0 and 500.0. Each order will have a unique ClOrderID. The following is a typical order.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 146
   MsgType (35): ORDER_SINGLE (D)
   SenderCompID (49): DLD_TEX
   TargetCompID (56): TEX_DLD
   MsgSeqNum (34): 499650
   SendingTime (52): 20181227-11:20:43
NewOrderSingle ("D")
   ClOrdID (11): ord509647-500000
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
   Symbol (55): BHP
   Side (54): BUY (1)
   TransactTime (60): 20181227-11:20:14
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.075
   TimeInForce (59): FILL_OR_KILL (4)
trailer ("trailer")
   CheckSum (10):

When the test server receives a NewOrderSingle message it sends either an ExecutionReport(8) with order reject, order cancel or order confirmation (randomly selected). The following is a typical order confirmation.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 224
   MsgType (35): EXECUTION_REPORT (8)
   SenderCompID (49): TEX_DLD
   TargetCompID (56): DLD_TEX
   MsgSeqNum (34): 1962926
   SendingTime (52): 20181227-11:22:07
ExecutionReport ("8")
   OrderID (37): ord499647
   ClOrdID (11): ord509647-500000
   ExecID (17): ord499647
   ExecTransType (20): NEW (0)
   ExecType (150): NEW (0)
   OrdStatus (39): NEW (0)
   Symbol (55): BHP
   Side (54): BUY (1)
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.07
   TimeInForce (59): FILL_OR_KILL (4)
   LastCapacity (29): 5
   LeavesQty (151): 57
   CumQty (14): 0
   AvgPx (6): 0
   TransactTime (60): 20181227-11:20:14
   ReportToExch (113): YES (Y)
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
   CheckSum (10):

Followed by one or more fills.

header ("header")
   BeginString (8): FIX.4.2
   BodyLength (9): 217
   MsgType (35): EXECUTION_REPORT (8)
   SenderCompID (49): TEX_DLD
   TargetCompID (56): DLD_TEX
   MsgSeqNum (34): 1962929
   SendingTime (52): 20181227-11:22:07
ExecutionReport ("8")
   OrderID (37): ord499647
   ClOrdID (11): ord509647-500000
   ExecID (17): exec1463279
   ExecTransType (20): NEW (0)
   ExecType (150): NEW (0)
   OrdStatus (39): FILLED (2)
   Symbol (55): BHP
   Side (54): BUY (1)
   OrderQty (38): 57
   OrdType (40): LIMIT (2)
   Price (44): 298.07
   TimeInForce (59): FILL_OR_KILL (4)
   LeavesQty (151): 0
   CumQty (14): 57
   AvgPx (6): 298.07
   TransactTime (60): 20181227-11:20:14
   HandlInst (21): AUTOMATED_EXECUTION_ORDER_PRIVATE_NO_BROKER_INTERVENTION (1)
trailer ("trailer")
   CheckSum (10):

The test client/server

Note that we didn't use tbb-malloc for this test, but used standard malloc()/free(). See the FAQ for details.

Before you begin, set your compiler optimisation to -O3 and probably to -march=native

% export CXXFLAGS=-O3 -march=native

You then need to pass the codec timing switch --enable-codectiming and the no fill message metadata switch
--enable-fillmetadata=no to configure and cleanly build Fix8.

% ./configure --prefix=[your target directory] --enable-codectiming --with-mpmc=tbb --enable-doxygen=no --enable-fillmetadata=no
% make clean; make install

If you have installed Fix8 from a release rpm, the supplied hftest will not be configured for this test. Please rebuild it following these instructions.

For our test, we will simulate an HF client preloading and sending 500000 orders to a server. We will measure the encode and decode timings for both the client and the server. We are also going to use the coroutine process mode, which will minimise the effects of jitter. The test client and server hftest are built by default when you build Fix8. Also by default, the xml configuration files hf_server.xml, hf_client.xml, hf_client_include.xml needed are supplied already preconfigured.

Do not use the configure switches shown above when building the other supplied test applications.

To run the server, execute the following command (assuming the xml config files are in the current directory):

% ./hftest -sol server

In another terminal, run the client:

% ./hftest -l client -p 150000 -u 50000 -c coro_hf_client.xml

We are telling the client you will be preloading 150000 messages. If you find the client disconnects during the test, restart the client in reliable mode:

% ./hftest -rl client -p 150000 -u 50000 -c coro_hf_client.xml
0 NewOrderSingle msgs currently preloaded.
loading...
0000001 A 2018-04-19 17:19:50.512934156 Starting session
0000002 A 2018-04-19 17:19:50.513103903 Trying to connect to: 127.0.0.1:11002 (1)
0000003 A 2018-04-19 17:19:50.513403209 Connection successful
0000004 A 2018-04-19 17:19:50.513421223 Session connected
0000005 B 2018-04-19 17:19:50.514467212 Client setting heartbeat interval to 10
0000006 B 2018-04-19 17:19:50.514470236 Heartbeat interval is 10
150000 NewOrderSingle msgs preloaded.

When the client has connected (there are no on screen messages), press ? to see the help menu...

Key     Command
===     =======
?       Help
N       Send n NewOrderSingle msgs
a       Send all Preloaded NewOrderSingle msgs
b       Batch preload and send n NewOrderSingle msgs
l       Logout
n       Send a NewOrderSingle msg
p       Preload n NewOrderSingle msgs
x       Exit

Now press 'a' to send the preloaded orders to the server:

150000 NewOrderSingle msgs sent
0 NewOrderSingle msgs remaining.
588567 ExecutionReport msgs received

When all the execution reports have been received (in the client window), press 'l' to logout and exit. Wait a few moments for the logging buffer to empty. When the client has disconnected the server will also flush its logging buffer so wait a bit longer for this as well. Occasionally you may get a core dump on exit - you can ignore this for now. Your server should have also now exited (we passed the o option).

You can use the supplied printer application hfprint to examine the protocol log output (you will need to enable the file protocol target in the xml configuration).

Preparing the results

If all has gone to plan you should have two log files, server and client. Examine the contents of these two files to see the results.

% cat client server|egrep "Encode|Decode"
client Encode:   185837 usecs, 150001 msgs,  1.238905074 usecs/msg, 0.81 msgs/usec, ...
client Decode:  2295686 usecs, 588567 msgs,  3.900466727 usecs/msg, 0.26 msgs/usec, ...
server Encode:   907882 usecs, 588567 msgs,  1.542529568 usecs/msg, 0.65 msgs/usec, ...
server Decode:   539200 usecs, 150001 msgs,  3.594642702 usecs/msg, 0.28 msgs/usec, ...

The results

Here are the tabulated results.

Mode	Operation	Msgs processed	Time µs	Msgs/µs	Average µs/msg
client	encode	150001	185837	0.81	1.23
	decode	588567	2295686	0.26	3.90
server	encode	588567	907882	0.65	1.54
	decode	150001	539200	0.28	3.59

Averaged across client and server...

Operation	Average µs/msg
encode	1.38
decode	3.75

Conclusion

Most users will be interested in client encode and decode. On our test hardware, Fix8 encodes a NewOrderSingle(D) in 1.38 µs and decodes an ExecutionReport(8) in 3.75 µs when compiled with gcc. It can also be seen that when averaged across client and server, encode performance is better than decode.

We believe performance is one of the distinguishing features of Fix8. Here we will present our current encode and decode performance stats along with a description of our test environment and test methodology.