report: rewrite subject contextualisation
Splitting it into sections makes it more readable, and goes toward padding the page count...
This commit is contained in:
parent
e18e70f0a9
commit
5f4d1f8125
107
report.md
107
report.md
|
@ -70,6 +70,18 @@ systems and related subjects, I was given this internship project to work on.
|
||||||
|
|
||||||
# Context of the subject
|
# Context of the subject
|
||||||
|
|
||||||
|
## Company trade
|
||||||
|
|
||||||
|
IMC, as its name suggests, is a market maker. It is specialised in providing
|
||||||
|
liquidity in the market by quoting both sides of the market, and profit off the
|
||||||
|
trades they make while providing this service.
|
||||||
|
|
||||||
|
One key ingredient to this business is latency: due to the competitive nature of
|
||||||
|
the market, we must process the incoming data and execute orders fast enough not
|
||||||
|
to get *picked off the market* with a bad position.
|
||||||
|
|
||||||
|
## Service
|
||||||
|
|
||||||
The exchange connectivity layer must route orders as fast possible, to stay
|
The exchange connectivity layer must route orders as fast possible, to stay
|
||||||
competitive, reduce transaction costs, and lower latencies which could result in
|
competitive, reduce transaction costs, and lower latencies which could result in
|
||||||
lost opportunities, therefore less profits.
|
lost opportunities, therefore less profits.
|
||||||
|
@ -80,6 +92,12 @@ conditional orders with this service: it must monitor the price of product A and
|
||||||
X, if product A's cost rise over X's, then it must start selling product B at
|
X, if product A's cost rise over X's, then it must start selling product B at
|
||||||
price Y.
|
price Y.
|
||||||
|
|
||||||
|
## The competition
|
||||||
|
|
||||||
|
FIXME: what can I even say about them?
|
||||||
|
|
||||||
|
## Strategy
|
||||||
|
|
||||||
A new exchange connectivity service, called the Execution Gateway, is being
|
A new exchange connectivity service, called the Execution Gateway, is being
|
||||||
built at IMC, the eventual goal being to migrate all trading strategies to using
|
built at IMC, the eventual goal being to migrate all trading strategies to using
|
||||||
this gateway to send orders to exchanges. This will allow it to be scaled more
|
this gateway to send orders to exchanges. This will allow it to be scaled more
|
||||||
|
@ -87,41 +105,66 @@ appropriately. However, care must be taken to maintain the current performance
|
||||||
during the entirety of the migration in order to stay competitive, and the only
|
during the entirety of the migration in order to stay competitive, and the only
|
||||||
way to ensure this is to measure it.
|
way to ensure this is to measure it.
|
||||||
|
|
||||||
With that context, let's review my expected tasks once more, and expand on each
|
## Roadmap
|
||||||
of them:
|
|
||||||
|
With that context, let's review my expected tasks once more, and expand on each
|
||||||
|
of them to get the roadmap:
|
||||||
|
|
||||||
|
* Become familiar with the service: before writing the code for the benchmark I
|
||||||
|
must first understand what goes into the process of a trade at IMC, what is
|
||||||
|
needed from the gateway and from the clients in order to run them and execute
|
||||||
|
orders. There is a lot of code at IMC: having different teams working at the
|
||||||
|
same time on different trading service results in a lot of churn. The global
|
||||||
|
execution team was created to centralise the work on core services that must be
|
||||||
|
provided to the rest of the IMC workforce. The global execution gateway is one
|
||||||
|
such project, aiming to consolidate all trading strategies under one singular
|
||||||
|
method to send orders to their exchanges.
|
||||||
|
|
||||||
* Become familiar with the service: before writing the code for the benchmark
|
|
||||||
I must first understand what goes into the process of a trade at IMC, what is
|
|
||||||
needed from the gateway and from the clients in order to run them and execute
|
|
||||||
orders. There is a lot of code at IMC: having different teams working at the
|
|
||||||
same time on different trading service results in a lot of churn. The
|
|
||||||
global execution team was created to centralise the work on core services that
|
|
||||||
must be provided to the rest of the IMC workforce. The global execution
|
|
||||||
gateway is one such project, aiming to consolidate all trading strategies
|
|
||||||
under one singular method to send orders to their exchanges.
|
|
||||||
* Write a dummy load generator: we want to send orders under different
|
* Write a dummy load generator: we want to send orders under different
|
||||||
conditions in order to run multiple scenarios which can model varying cases
|
conditions in order to run multiple scenarios which can model varying cases of
|
||||||
of execution. Having more data for varying corner cases can make us more
|
execution. Having more data for varying corner cases can make us more confident
|
||||||
confident of the robustness and efficiency of the service. This is especially
|
of the robustness and efficiency of the service. This is especially needed
|
||||||
needed becaue of the various roles that the gateway must fulfill: not only
|
becaue of the various roles that the gateway must fulfill: not only must it act
|
||||||
must it act as a bridge for the communication between exchanges and traders,
|
as a bridge for the communication between exchanges and traders, but also as an
|
||||||
but also as an order executor. All those cases must be accounted for when
|
order executor. All those cases must be accounted for when writing the different
|
||||||
writing the different scenarios.
|
scenarios.
|
||||||
|
|
||||||
* Benchmark the system under the load: once we can run those scenarios smoothly
|
* Benchmark the system under the load: once we can run those scenarios smoothly
|
||||||
we can start taking multiple measurements. The main one that IMC is interested
|
we can start taking multiple measurements. The main one that IMC is interested
|
||||||
in is wall-to-wall latency (abbreviated W2W): the time it takes for a trade to
|
in is wall-to-wall latency (abbreviated W2W): the time it takes for a trade to
|
||||||
go from a trading strategy to an exchange. The lower this time, the more
|
go from a trading strategy to an exchange. The lower this time, the more
|
||||||
occasions there are to make good trades.
|
occasions there are to make good trades. FIXME: probably more context in my
|
||||||
FIXME: probably more context in my notes
|
notes
|
||||||
|
|
||||||
* Analyze the measurements: the global execution team has some initial
|
* Analyze the measurements: the global execution team has some initial
|
||||||
expectations of the gateway's performance. A divergence on that part could
|
expectations of the gateway's performance. A divergence on that part could mean
|
||||||
mean that the measurements are flawed in some way, or that the gateway is not
|
that the measurements are flawed in some way, or that the gateway is not
|
||||||
performing as expected. Further analysis can be done to look at the difference
|
performing as expected. Further analysis can be done to look at the difference
|
||||||
between mean execution time and the 99th percentile, and analyse the tail of
|
between mean execution time and the 99th percentile, and analyse the tail of the
|
||||||
the timing distribution: the smaller it is the better. Consistent timing is
|
timing distribution: the smaller it is the better. Consistent timing is more
|
||||||
more important than a lower average, because we must be absolutely confident
|
important than a lower average, because we must be absolutely confident that a
|
||||||
that a trade order is going to be executed smoothly, and introducing
|
trade order is going to be executed smoothly, and introducing inconsistent
|
||||||
inconsistent latency can result in bad trades.
|
latency can result in bad trades.
|
||||||
|
|
||||||
|
## Internship positioning amongst company works
|
||||||
|
|
||||||
|
My work was focused on providing a framework to instrument gateways under
|
||||||
|
different scenarios.
|
||||||
|
|
||||||
|
Once that framework is built, to be effective it must be integrated in the
|
||||||
|
existing Continuous Integration platform used at IMC. This enables us to track
|
||||||
|
breaking changes and, eventually, be notified of performance regressions.
|
||||||
|
That last part is yet to be done, needing to be integrated with the new change
|
||||||
|
point detection tool currently being developed internally. Once that is done, we
|
||||||
|
can feed the performance results to automatically see when a regression has been
|
||||||
|
introduced into the system.
|
||||||
|
|
||||||
|
With the knowledge I gained working on this project, my next task was to add
|
||||||
|
compatibility testing to ensure backward and forward compatibility of the
|
||||||
|
clients and gateways. This meant having to run the existing tests using the
|
||||||
|
actual production binaries of the gateway, and making sure the tests keep
|
||||||
|
working across versions. This is very similar to the way the benchmarks work,
|
||||||
|
and I could reuse most of the tools developed for the framework to that end.
|
||||||
|
|
||||||
# Internship roadmap
|
# Internship roadmap
|
||||||
|
|
||||||
|
@ -147,6 +190,8 @@ I wrote and learned during the previous month.
|
||||||
|
|
||||||
# Engineering practices
|
# Engineering practices
|
||||||
|
|
||||||
|
Problematic: development of a benchmark framework
|
||||||
|
|
||||||
# Illustrated analysis of acquired skills
|
# Illustrated analysis of acquired skills
|
||||||
|
|
||||||
# Added value
|
# Added value
|
||||||
|
|
Loading…
Reference in a new issue