report: add exec sum section
This commit is contained in:
parent
53f34563bc
commit
0577e1d911
90
report.md
90
report.md
|
@ -1,5 +1,95 @@
|
|||
# Exec Sum
|
||||
|
||||
When making my decision to major in Image Processing and Image Synthesis, my
|
||||
main motivator was my growing curiosity for high performance programming, as I
|
||||
thought that some of the subjects being taught would lend themselves well to
|
||||
furthering my understanding of this domain.\
|
||||
On my own, I discovered through a number of conference talks that I watched that
|
||||
the field of finance provides many interesting challenges that aligned with
|
||||
those interests.
|
||||
|
||||
When the opportunity arose to work for IMC, a leading firm in the world of
|
||||
*market-making*, I jumped at the chance to apply to them for my internship. When
|
||||
asked for the kind of work I wanted to do during that time, I highlighted my
|
||||
interests in performance, which lead to the subject of writing a benchmark
|
||||
framework for their new exchange connectivity layer, currently in the process of
|
||||
being created and deployed.\
|
||||
This felt like the perfect subject to learn more about finance, a field I had
|
||||
been interested for some time at that point, and get exposed to people that are
|
||||
knowledgeable, to cultivate my budding interest in the field.
|
||||
|
||||
During my internship, I was part of the *Global Execution* team at IMC, a new
|
||||
team that was tasked with working on projects of relevance to multiple desks
|
||||
(each of whom is dedicated to working on a specific exchange). Their first
|
||||
contribution being to write the *Execution Gateway*, a piece of software that
|
||||
should act as the intermediary between IMC's trading algorithm and external
|
||||
exchanges. This is meant to replace the current pattern of each auto-trader
|
||||
connecting directly to the exchanges to place orders, migrating instead to a
|
||||
central piece of infrastructure to do the translation between internal IMC
|
||||
protocols and decision, and exchange-facing requests.
|
||||
|
||||
As part of this migration process, my mission was to work on providing a simple
|
||||
framework that could be used to measure the performance of such a gateway. To do
|
||||
so, we must be able to instrument one under various scenarios meant to mirror
|
||||
real-life conditions, or exercise edge-cases.
|
||||
|
||||
This lead me to first get acquainted with the components that go into running
|
||||
the gateway, and what is necessary on the client side to make use of it through
|
||||
the *Execution API*, which is in the interface exposed to downstream consumers
|
||||
of the gateway: the trading algorithms.\
|
||||
Similarly, I learned what the gateway itself needs to be able to communicate
|
||||
with an exchange, as I would also need to control that side in the framework to
|
||||
allow measurements of the time taken for an order from the client to the
|
||||
exchange.\
|
||||
To learn about those dependencies, I studied the existing tests for the
|
||||
*Execution API*, which bundle client, gateway, and exchange in a single process
|
||||
to test their correct behaviour.
|
||||
|
||||
Once I got more familiar with each piece of the puzzle, I got to work on writing
|
||||
the framework itself. This required writing a few modules that would provide
|
||||
data needed as input by the gateway as a pre-requisite to being able to run
|
||||
anything. With those done, getting every component running properly was the next
|
||||
step. With that done, the last part was to generate some dummy load on the
|
||||
gateway, and collecting its performance measurements. Those could be analyzed
|
||||
offline, in a more exploratory manner.\
|
||||
With that work done, I had delivered my final product relating to my internship
|
||||
subject. Taken from a first *Proof-Of-Concept* to a working end result. Along
|
||||
the way I learned about the specific needs relating to the usage of those
|
||||
benchmarks, and integrated them in the greater IMC ecosystem.
|
||||
|
||||
With the initial work on the benchmark framework done, my next area of focus was
|
||||
to add compatibility testing for the gateway. The point of this work was to add,
|
||||
as part of the *Continuous Integration* pipeline in use at IMC, tests that would
|
||||
exercise the actual gateway binaries used in production, to test for both
|
||||
forward and backward compatibility. This work fits perfectly as a continuation
|
||||
of what I did to write the benchmark framework: both need to run the gateway and
|
||||
generate specific scenarios to test its behaviour. This allowed me to reuse most
|
||||
of the code what I had written for the benchmark, and apply it to writing the
|
||||
tests.\
|
||||
The need for reliable tests meant that I had to do a lot of ground work to
|
||||
ensure that they were not flaky, this is probably the part that took longest in
|
||||
the process, with some deep investigations to understand some subtle bugs and
|
||||
behaviours that were exposed by the new tests I was attempting to integrate.
|
||||
|
||||
Towards the end of my internship, I presented the work I did on the framework to
|
||||
other developers in the execution teams. This was in part to showcase the work
|
||||
being done by the *Global Execution* team, and to participate in the regular
|
||||
knowledge sharing that happens at IMC.
|
||||
|
||||
Joining a company during the COVID period of quarantines, working-from-home, and
|
||||
the relatively low amount of face-to-face contact high-lighted the need for
|
||||
efficient ways of communicating with my colleagues. Being part of a productive,
|
||||
highly driven team has been a pleasure.
|
||||
|
||||
Overall, this internship was a great reaffirmation of my career choices. This
|
||||
was my first experience with an actual project dealing with performance of
|
||||
something actually used in production, and not simply a one-off project to work
|
||||
on and leave aside. Being able to contribute something which will be used
|
||||
long-term, and have impactful consequences, reassures me in the choice of being
|
||||
a software engineer, and the impact of my studies at EPITA.
|
||||
|
||||
\newpage
|
||||
|
||||
# Thanks and acknowledgements
|
||||
|
||||
First off, I would like to thank Jelle Wissink, an engineer from the Global
|
||||
|
|
Loading…
Reference in a new issue