ambroisie/report
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

report: add engineering practices section

Browse source
This commit is contained in:
Bruno BELANYI 2021-07-22 20:27:41 +02:00
parent 47bf3793fc
commit 568ba04ce7
1 changed files with 109 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

110
report.md
View file

@ -323,7 +323,115 @@ FIXME
# Engineering practices
Problematic: development of a benchmark framework
## Delivering a project from scratch to completion
During the course of my internship, I have had to deliver a product from its
first *Proof of Concept* to a usable deliverable, going through various
iterations along the way.
This process started with me getting familiar with the IMC code base, coming up
to speed with the tooling in use, some of the knowledge needed to work on the
existing code, etc... Once I was productive, I could start exploring the
necessary component needed to write a benchmark using the gateway binaries. This
initial POC was written by gutting some existing code in use for the Execution
API tests, splitting it into client and exchange code on one side, and gateway
code on the other.
From this initial experiment, I got more familiar with the different components
that would go into an implementation of the benchmark. With the help of my
mentor, I could identify the needed dependencies that would to be provided to
the gateway binary in order to instrument it under different scenarios.
I worked on writing those components in a way that was usable for the benchmark,
making sure that they were working an tested along the way. One such component
was writing a fake version of the RDS (FIXME: what does it mean again?) that
would be populated from the benchmark scenario, and provided the information
about financial instruments to the gateway in order to use them in the scenario,
e.g: ordering a stock.
I went on to write a first version of the benchmark framework for a specific
gateway and a specific exchange: this served as the basis for further iteration
after receiving feedback about my design. Writing a second benchmark for a
second exchange and gateway lead to more re-design.
The basic components of the benchmark framework were useful outside of their
original intended purposes, as I could reuse them to write the compatibility
tests, which were similar in the way they make use of the gateway to run a
scenario and verify the correct behaviour of our system.
With all that work accomplished, I now needed to share my knowledge to make sure
that my mentor and I were not the only people who knew that this framework
existed, how to use, or how to work with it. To that end, I documented by
writing some useful tips I could give to help debug the benchmarks and tests
using gateway binaries. I also gave a presentation at the end of my internship
to demonstrate how to run a benchmark, and explain the main components of the
framework.
I have delivered a complete, featureful product from scratch to finish, complete
with documentation and demonstration of its use. This is at the heart of our
schooling at EPITA: making us well-rounded engineers that can deliver their work
to completion. FIXME: wax a bit more poetics
## Acquiring new skills and knowledge
IMC is part of the financial tech sector, taking a position of market-maker on a
large amount of exchanges world-wide.
The financial sector, even though I was attracted to it by my previous exposure
from conference sponsors and blog posts from engineers in the sector, was still
something I was not familiar with when first joining the company.
There is a large amount of vocabulary and knowledge specific to this industry,
not even to mention the infrastructure and tooling in use at IMC, which while
not specific to this industry in particular, was still foreign to me at that
point.
Before starting my internship, I was advised to read a book about high frequency
trading, which gave me some context on how exchanges work, and a few important
words that are part of the financial vocabulary. In addition, I learned about
IMC's trading infrastructure through a number of presentations that my team lead
organised with new hires during the beginning of my internship. This not only
gave me more context about what part of the existing infrastructure was aimed to
be replaced by the new *Execution Gateway* and the *Execution API*. I also got
to learn about some of the basics of pricing theory, which underpins our whole
strategy layer to come up with an appropriate valuation for any product we are
interested in trading.
I got to further learn about trading and option theory during a training week
organized with a dozen other summer interns: we were taught some of the
mathematics that form the basics of valuation reducing risks in trading, the
associated vocabulary, and apply them during workshop exercises in trading with
the other interns.
On the technical side, not only did I learn about the software stack in use at
IMC, as I worked on more and more parts of the code base I discovered new
tooling put in place to work and debug parts of our stack that are too costly to
setup or use on any dev computer. One such solution is the *fullsim* system,
which allows us to simulate our FPGA engines in software, to allow developers to
debug issues with FPGA interaction without having to fiddle with actual FPGA
cards or know how to use them. I also introduced my colleagues to new tools that
they were unaware of, the most prominent being the one I always reach for first
when trying to debug a piece of software: `rr`, which allows one to record a
program's execution and run it under a debugger in a totally deterministic
manner: it allows replaying and rewinding execution at will, making it a great
asset when dealing with issues that are sporadic, or require tricky timings like
networked systems.
IMC encourages knowledge sharing across all teams, it permeates the company
culture, and shows in many ways. An execution engineer is encouraged to learn
about trading, which gives us more context when interacting with traders,
spotting mistakes in new strategies or guiding which features would make sense
to write next. Catch-up meetings are organized regularly between teams.
Presentations are given to teach people about the work that is being
accomplished to improve every part of our infrastructure, from deployment
tooling, developer productivity, to new strategies or components of our systems.
Thanks to my well-rounded education, not only do I feel comfortable being
exposed to all this information. But I have felt confident that I fit in from
the start, and could keep pace with the information that was fed to me. I am
able to pick up those new skills fast, because EPITA taught us the most
important skill of this trade: I learnt how to learn, and how to flourish while
doing so.
# Illustrated analysis of acquired skills