How to write a research paper: a perspective.

This post is about how to write a research paper (oh boy, starting the blog off with a rant). Over the course of the past few years I have read well upward of 60 papers in each of Operating Systems, Distributed Systems, Computer/Datacenter Networking and Computer Architecture. Yes, this is boasting, I know, but it has two advantages:

1) It helps add weight to my argument.

2) If you're reading this probably you are a recruiter collecting information about me in about 3-5 years time (there is otherwise no reason anyone would read a random technical blog), and so I should try to impress you (or should I?)

After reading these papers, I have found that the "paper-writing" (research) industry is plagued with bad practices. Almost all courses involving reading papers start with a paper titled "How to read a paper", yet we do not teach how to WRITE one (maybe we do, but anyway...)

Thus I have decided to make the job of writing a paper slightly easier by telling you (and anyone who writes a paper) how to write a paper. Here goes.

1) Dont. This sure seems like a rude advice: after all why wouldn't one write papers? But judging by the ones I've read, at least 5-10% of the ones written offer nothing new. Again, this is rude. But its true. Often a paper is about some system which is a re-implementation of some previous work(s) with tiny tweaks that are hacks more than they are innovations. Such papers are far better suited for technical articles, save it for the articles.

2) Please publish the code of your benchmarks. I am not asking to get the code of the work published; this can be up to the authors. However, when you claim that your system gained a 3.2X improvement over system XYZ on a workload consisting of say a Zipf distribution of writes/reads on n servers, make sure you publish the code that generated this workload: it makes both replication and extension of your work easier.

3) Please report truthful values! Now I admit that maybe 99.9% papers report correct values. But 0.1% is still too much. True story: I read a paper about model Y which claimed 8.2X speedup over model X. Later I read of a model Z claiming 4x improvement over Y. However it turns out that "in practice" X performs better than Z. So clearly either the workload was skewed or one of the people is lying. Dont do this.

4) Do not dwell too much on the API: this is not a user manual! Yes, I have seen this often, too often. Frequently I pick up a paper some 14-15 pages long and 3-4 of them are just about what functions are exposed by the system. Admittedly sometimes this is required, as in what functions to implement over a persistent key-value store (append or arbitrary write etc.) but that's not what I'm referring to here: I have seen cases where the authors actually tell the reader how to use the <whatever>, complete with example code and the like. Save this for your user documentation. If your model is good people will undoubtedly read the manual and be interested.

5) (Try to) have a takeaway. Now this one is more subjective. But honestly, the research community as a whole is (I think) more interested and helped by what advancements to the state-of-art one can make. Sure, implementing a large scale system with lots of features is no small feat, but to me what makes a paper interesting/useful is key insight/takeaways. These are the things that will stay relevant much later on. Here is an example: this paper: End-to-End arguments in System Design had a clear takeaway, way back in 1981:

" Using performance to justify placing functions in a low-level subsystem must be done carefully.Sometimes, by examining the problem thoroughly, the same or better performance enhancement can be achieved at the high level. "

38 years later, in NSDI 2019, a paper winning the best paper award was the following: Datacenter RPCs can be General and Fast. The key "point" of this paper built upon this exact takeaway above (word-to-word; dont believe me? check for yourself!). Now this is an example of good research: such key insights maintain relevance for more than 35 years!

6) Keep your paper short. This is the final and most important one. I dare say that over 80% of papers I have read explain something in 15+ pages what they could in 2-3. And that is not coincidental. People tend to repeat information multiple times in a paper, once each in "Abstract", "Introduction", "Model", "Discussion", "Conclusion" and so forth. It often gets to a point where it gets borderline boring to read (trust me when I say I'm one who enjoys reading papers). While there is unspoken consensus that the language used in papers must be formal and to-the-point there is no such agreement that papers should not be over-extended. Yes, maybe it is harder to get a 4-page paper accepted compared to 15, and maybe that is why this happens. But trust me, no one will enjoy reading the same thing in 15 pages instead of 4.

And that is all! It was a grumpy little rant and probably disrespectful to the many who have poured hours into writing that one paper. Yet, I believe there is a grain of truth in what I've written. Hopefully I made you rethink how to write your next paper.