report: add engineering practices section
This commit is contained in:
parent
47bf3793fc
commit
568ba04ce7
110
report.md
110
report.md
|
@ -323,7 +323,115 @@ FIXME
|
||||||
|
|
||||||
# Engineering practices
|
# Engineering practices
|
||||||
|
|
||||||
Problematic: development of a benchmark framework
|
## Delivering a project from scratch to completion
|
||||||
|
|
||||||
|
During the course of my internship, I have had to deliver a product from its
|
||||||
|
first *Proof of Concept* to a usable deliverable, going through various
|
||||||
|
iterations along the way.
|
||||||
|
|
||||||
|
This process started with me getting familiar with the IMC code base, coming up
|
||||||
|
to speed with the tooling in use, some of the knowledge needed to work on the
|
||||||
|
existing code, etc... Once I was productive, I could start exploring the
|
||||||
|
necessary component needed to write a benchmark using the gateway binaries. This
|
||||||
|
initial POC was written by gutting some existing code in use for the Execution
|
||||||
|
API tests, splitting it into client and exchange code on one side, and gateway
|
||||||
|
code on the other.
|
||||||
|
|
||||||
|
From this initial experiment, I got more familiar with the different components
|
||||||
|
that would go into an implementation of the benchmark. With the help of my
|
||||||
|
mentor, I could identify the needed dependencies that would to be provided to
|
||||||
|
the gateway binary in order to instrument it under different scenarios.
|
||||||
|
|
||||||
|
I worked on writing those components in a way that was usable for the benchmark,
|
||||||
|
making sure that they were working an tested along the way. One such component
|
||||||
|
was writing a fake version of the RDS (FIXME: what does it mean again?) that
|
||||||
|
would be populated from the benchmark scenario, and provided the information
|
||||||
|
about financial instruments to the gateway in order to use them in the scenario,
|
||||||
|
e.g: ordering a stock.
|
||||||
|
|
||||||
|
I went on to write a first version of the benchmark framework for a specific
|
||||||
|
gateway and a specific exchange: this served as the basis for further iteration
|
||||||
|
after receiving feedback about my design. Writing a second benchmark for a
|
||||||
|
second exchange and gateway lead to more re-design.
|
||||||
|
|
||||||
|
The basic components of the benchmark framework were useful outside of their
|
||||||
|
original intended purposes, as I could reuse them to write the compatibility
|
||||||
|
tests, which were similar in the way they make use of the gateway to run a
|
||||||
|
scenario and verify the correct behaviour of our system.
|
||||||
|
|
||||||
|
With all that work accomplished, I now needed to share my knowledge to make sure
|
||||||
|
that my mentor and I were not the only people who knew that this framework
|
||||||
|
existed, how to use, or how to work with it. To that end, I documented by
|
||||||
|
writing some useful tips I could give to help debug the benchmarks and tests
|
||||||
|
using gateway binaries. I also gave a presentation at the end of my internship
|
||||||
|
to demonstrate how to run a benchmark, and explain the main components of the
|
||||||
|
framework.
|
||||||
|
|
||||||
|
I have delivered a complete, featureful product from scratch to finish, complete
|
||||||
|
with documentation and demonstration of its use. This is at the heart of our
|
||||||
|
schooling at EPITA: making us well-rounded engineers that can deliver their work
|
||||||
|
to completion. FIXME: wax a bit more poetics
|
||||||
|
|
||||||
|
## Acquiring new skills and knowledge
|
||||||
|
|
||||||
|
IMC is part of the financial tech sector, taking a position of market-maker on a
|
||||||
|
large amount of exchanges world-wide.
|
||||||
|
|
||||||
|
The financial sector, even though I was attracted to it by my previous exposure
|
||||||
|
from conference sponsors and blog posts from engineers in the sector, was still
|
||||||
|
something I was not familiar with when first joining the company.
|
||||||
|
|
||||||
|
There is a large amount of vocabulary and knowledge specific to this industry,
|
||||||
|
not even to mention the infrastructure and tooling in use at IMC, which while
|
||||||
|
not specific to this industry in particular, was still foreign to me at that
|
||||||
|
point.
|
||||||
|
|
||||||
|
Before starting my internship, I was advised to read a book about high frequency
|
||||||
|
trading, which gave me some context on how exchanges work, and a few important
|
||||||
|
words that are part of the financial vocabulary. In addition, I learned about
|
||||||
|
IMC's trading infrastructure through a number of presentations that my team lead
|
||||||
|
organised with new hires during the beginning of my internship. This not only
|
||||||
|
gave me more context about what part of the existing infrastructure was aimed to
|
||||||
|
be replaced by the new *Execution Gateway* and the *Execution API*. I also got
|
||||||
|
to learn about some of the basics of pricing theory, which underpins our whole
|
||||||
|
strategy layer to come up with an appropriate valuation for any product we are
|
||||||
|
interested in trading.
|
||||||
|
|
||||||
|
I got to further learn about trading and option theory during a training week
|
||||||
|
organized with a dozen other summer interns: we were taught some of the
|
||||||
|
mathematics that form the basics of valuation reducing risks in trading, the
|
||||||
|
associated vocabulary, and apply them during workshop exercises in trading with
|
||||||
|
the other interns.
|
||||||
|
|
||||||
|
On the technical side, not only did I learn about the software stack in use at
|
||||||
|
IMC, as I worked on more and more parts of the code base I discovered new
|
||||||
|
tooling put in place to work and debug parts of our stack that are too costly to
|
||||||
|
setup or use on any dev computer. One such solution is the *fullsim* system,
|
||||||
|
which allows us to simulate our FPGA engines in software, to allow developers to
|
||||||
|
debug issues with FPGA interaction without having to fiddle with actual FPGA
|
||||||
|
cards or know how to use them. I also introduced my colleagues to new tools that
|
||||||
|
they were unaware of, the most prominent being the one I always reach for first
|
||||||
|
when trying to debug a piece of software: `rr`, which allows one to record a
|
||||||
|
program's execution and run it under a debugger in a totally deterministic
|
||||||
|
manner: it allows replaying and rewinding execution at will, making it a great
|
||||||
|
asset when dealing with issues that are sporadic, or require tricky timings like
|
||||||
|
networked systems.
|
||||||
|
|
||||||
|
IMC encourages knowledge sharing across all teams, it permeates the company
|
||||||
|
culture, and shows in many ways. An execution engineer is encouraged to learn
|
||||||
|
about trading, which gives us more context when interacting with traders,
|
||||||
|
spotting mistakes in new strategies or guiding which features would make sense
|
||||||
|
to write next. Catch-up meetings are organized regularly between teams.
|
||||||
|
Presentations are given to teach people about the work that is being
|
||||||
|
accomplished to improve every part of our infrastructure, from deployment
|
||||||
|
tooling, developer productivity, to new strategies or components of our systems.
|
||||||
|
|
||||||
|
Thanks to my well-rounded education, not only do I feel comfortable being
|
||||||
|
exposed to all this information. But I have felt confident that I fit in from
|
||||||
|
the start, and could keep pace with the information that was fed to me. I am
|
||||||
|
able to pick up those new skills fast, because EPITA taught us the most
|
||||||
|
important skill of this trade: I learnt how to learn, and how to flourish while
|
||||||
|
doing so.
|
||||||
|
|
||||||
# Illustrated analysis of acquired skills
|
# Illustrated analysis of acquired skills
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue