report: rewrite subject contextualisation
Splitting it into sections makes it more readable, and goes toward padding the page count...
This commit is contained in:
parent
e18e70f0a9
commit
5f4d1f8125
107
report.md
107
report.md
|
@ -70,6 +70,18 @@ systems and related subjects, I was given this internship project to work on.
|
|||
|
||||
# Context of the subject
|
||||
|
||||
## Company trade
|
||||
|
||||
IMC, as its name suggests, is a market maker. It is specialised in providing
|
||||
liquidity in the market by quoting both sides of the market, and profit off the
|
||||
trades they make while providing this service.
|
||||
|
||||
One key ingredient to this business is latency: due to the competitive nature of
|
||||
the market, we must process the incoming data and execute orders fast enough not
|
||||
to get *picked off the market* with a bad position.
|
||||
|
||||
## Service
|
||||
|
||||
The exchange connectivity layer must route orders as fast possible, to stay
|
||||
competitive, reduce transaction costs, and lower latencies which could result in
|
||||
lost opportunities, therefore less profits.
|
||||
|
@ -80,6 +92,12 @@ conditional orders with this service: it must monitor the price of product A and
|
|||
X, if product A's cost rise over X's, then it must start selling product B at
|
||||
price Y.
|
||||
|
||||
## The competition
|
||||
|
||||
FIXME: what can I even say about them?
|
||||
|
||||
## Strategy
|
||||
|
||||
A new exchange connectivity service, called the Execution Gateway, is being
|
||||
built at IMC, the eventual goal being to migrate all trading strategies to using
|
||||
this gateway to send orders to exchanges. This will allow it to be scaled more
|
||||
|
@ -87,41 +105,66 @@ appropriately. However, care must be taken to maintain the current performance
|
|||
during the entirety of the migration in order to stay competitive, and the only
|
||||
way to ensure this is to measure it.
|
||||
|
||||
With that context, let's review my expected tasks once more, and expand on each
|
||||
of them:
|
||||
## Roadmap
|
||||
|
||||
With that context, let's review my expected tasks once more, and expand on each
|
||||
of them to get the roadmap:
|
||||
|
||||
* Become familiar with the service: before writing the code for the benchmark I
|
||||
must first understand what goes into the process of a trade at IMC, what is
|
||||
needed from the gateway and from the clients in order to run them and execute
|
||||
orders. There is a lot of code at IMC: having different teams working at the
|
||||
same time on different trading service results in a lot of churn. The global
|
||||
execution team was created to centralise the work on core services that must be
|
||||
provided to the rest of the IMC workforce. The global execution gateway is one
|
||||
such project, aiming to consolidate all trading strategies under one singular
|
||||
method to send orders to their exchanges.
|
||||
|
||||
* Become familiar with the service: before writing the code for the benchmark
|
||||
I must first understand what goes into the process of a trade at IMC, what is
|
||||
needed from the gateway and from the clients in order to run them and execute
|
||||
orders. There is a lot of code at IMC: having different teams working at the
|
||||
same time on different trading service results in a lot of churn. The
|
||||
global execution team was created to centralise the work on core services that
|
||||
must be provided to the rest of the IMC workforce. The global execution
|
||||
gateway is one such project, aiming to consolidate all trading strategies
|
||||
under one singular method to send orders to their exchanges.
|
||||
* Write a dummy load generator: we want to send orders under different
|
||||
conditions in order to run multiple scenarios which can model varying cases
|
||||
of execution. Having more data for varying corner cases can make us more
|
||||
confident of the robustness and efficiency of the service. This is especially
|
||||
needed becaue of the various roles that the gateway must fulfill: not only
|
||||
must it act as a bridge for the communication between exchanges and traders,
|
||||
but also as an order executor. All those cases must be accounted for when
|
||||
writing the different scenarios.
|
||||
conditions in order to run multiple scenarios which can model varying cases of
|
||||
execution. Having more data for varying corner cases can make us more confident
|
||||
of the robustness and efficiency of the service. This is especially needed
|
||||
becaue of the various roles that the gateway must fulfill: not only must it act
|
||||
as a bridge for the communication between exchanges and traders, but also as an
|
||||
order executor. All those cases must be accounted for when writing the different
|
||||
scenarios.
|
||||
|
||||
* Benchmark the system under the load: once we can run those scenarios smoothly
|
||||
we can start taking multiple measurements. The main one that IMC is interested
|
||||
in is wall-to-wall latency (abbreviated W2W): the time it takes for a trade to
|
||||
go from a trading strategy to an exchange. The lower this time, the more
|
||||
occasions there are to make good trades.
|
||||
FIXME: probably more context in my notes
|
||||
we can start taking multiple measurements. The main one that IMC is interested
|
||||
in is wall-to-wall latency (abbreviated W2W): the time it takes for a trade to
|
||||
go from a trading strategy to an exchange. The lower this time, the more
|
||||
occasions there are to make good trades. FIXME: probably more context in my
|
||||
notes
|
||||
|
||||
* Analyze the measurements: the global execution team has some initial
|
||||
expectations of the gateway's performance. A divergence on that part could
|
||||
mean that the measurements are flawed in some way, or that the gateway is not
|
||||
performing as expected. Further analysis can be done to look at the difference
|
||||
between mean execution time and the 99th percentile, and analyse the tail of
|
||||
the timing distribution: the smaller it is the better. Consistent timing is
|
||||
more important than a lower average, because we must be absolutely confident
|
||||
that a trade order is going to be executed smoothly, and introducing
|
||||
inconsistent latency can result in bad trades.
|
||||
expectations of the gateway's performance. A divergence on that part could mean
|
||||
that the measurements are flawed in some way, or that the gateway is not
|
||||
performing as expected. Further analysis can be done to look at the difference
|
||||
between mean execution time and the 99th percentile, and analyse the tail of the
|
||||
timing distribution: the smaller it is the better. Consistent timing is more
|
||||
important than a lower average, because we must be absolutely confident that a
|
||||
trade order is going to be executed smoothly, and introducing inconsistent
|
||||
latency can result in bad trades.
|
||||
|
||||
## Internship positioning amongst company works
|
||||
|
||||
My work was focused on providing a framework to instrument gateways under
|
||||
different scenarios.
|
||||
|
||||
Once that framework is built, to be effective it must be integrated in the
|
||||
existing Continuous Integration platform used at IMC. This enables us to track
|
||||
breaking changes and, eventually, be notified of performance regressions.
|
||||
That last part is yet to be done, needing to be integrated with the new change
|
||||
point detection tool currently being developed internally. Once that is done, we
|
||||
can feed the performance results to automatically see when a regression has been
|
||||
introduced into the system.
|
||||
|
||||
With the knowledge I gained working on this project, my next task was to add
|
||||
compatibility testing to ensure backward and forward compatibility of the
|
||||
clients and gateways. This meant having to run the existing tests using the
|
||||
actual production binaries of the gateway, and making sure the tests keep
|
||||
working across versions. This is very similar to the way the benchmarks work,
|
||||
and I could reuse most of the tools developed for the framework to that end.
|
||||
|
||||
# Internship roadmap
|
||||
|
||||
|
@ -147,6 +190,8 @@ I wrote and learned during the previous month.
|
|||
|
||||
# Engineering practices
|
||||
|
||||
Problematic: development of a benchmark framework
|
||||
|
||||
# Illustrated analysis of acquired skills
|
||||
|
||||
# Added value
|
||||
|
|
Loading…
Reference in a new issue