Writing with Scott’s Group

Write clearly

• Prototype and share a sketch of your argument before fleshing out a full draft.This will help you focus your efforts, and make sure that your activities align with your goals. It's a real bummer to realize after the fact that you spent three months building something irrelevant. Or that you spent two months running an experiment that doesn't show what you wanted it to. Chinmay and Ed Chi have an implementation of the outline-early strategy that I strongly recommend: write your outline as a slide deck and give it as a talk. They point out that this forces you to articulate what's important and focus on it. Also, slide software is provides a good interface for rearranging content.
• Write early and often: The best way to increase the odds that your paper is rejected: do most of the writing at the last minute. The best way to increase the odds of acceptance: have a complete and coherent draft several weeks in advance; solicit and incorporate feedback from other PhD students in the group. Your peers can serve as 'pilot' readers so you get the kinks and errors out of your paper before reviewers see it. If you don't solicit and incorporate feedback, the reviewers will get confused, find errors and reject your paper. There's a saying, "an ounce of prevention is worth a pound of cure" and it's true here in a big way. A small amount of up-front discipline and preparation can hugely increase odds of acceptance. By contrast, getting rejected likely delays publication by a year – and a year is a long time to unnecessarily have a big incomplete goal occupying your time and attention – you also risk getting scooped, made less relevant, or needing to rework your framing based on publications that appear in the meantime. Given all the stress and work of those downsides, don't you think it's a good idea to draft your paper early and focus on writing well?
• Milestone documents you write will, in all likelihood, take much longer to write than you plan for. Milestone documents are things like your first paper, thesis proposal, job research statement, job talk, dissertation, and (when starting faculty life) NSF CAREER proposal. Hofstadter's Law. means you can't know how long these will take. (The answer for most of these is a month of working on it every day from the time you start till you're done, but you won't really believe that when reading this — or it least will behave like you don't — even though it's true.) These are also things you really don't want to screw up. So please don't think: "it'll take x days, so I'll start x days before I need to get it to my committee / send it off." Put a buffer in there.
• Providing meaningful feedback on writing takes a significant contiguous block of time. While I try really hard to get comments to you as quickly as I can, you should expect that it will take me 1-2 weeks to get you feedback on a paper draft. Sometimes I may be faster. And when I have a major upcoming deadline it may be slower; I may not always be able to get you feedback as quickly as you or I would like. Also, let me know what feedback you would like. In general, my editing will include careful reworking of sentences, detailed changes to graphs, etc. If your paper isn't ready for that yet, and you'd only like structural feedback (i.e., you'll send me a version for detailed editing later), please let me know. That way, neither of us spends time 'getting the design right' before we've decided on the 'right design.'
• Address all edits: The edits I make are not optional suggestions that you can ignore if you wish. For every single edit that I mark, either make the edit or add a comment explaining why you didn't make it or did something differently. Dialogue is essential: if you disagree with an edit (e.g., if my revision would be incorrect or read poorly), don't just leave the text as it is. Think about what motivated me to try and revise it. And propose an alternative version that is more accurate, clearer, …
• Write durable, impactful papers. I suggest a MiniMax strategy: in order to cover maximal terrain, embed minimal assumptions in the writing. You want as many people as possible to get something out of your paper. So don't make it too domain specific. Imagine people reading it a decade from now. Write it so that it will still feel relevant, and not dated. Talks and videos, on the other hand, are more ephemeral, and consequently its more appropriate to tie them more strongly to the time and the domain.
  Related to the MiniMax strategy is Stu Card's weak methods/strong methods strategy. This is useful for both both system building and writing. Have a general ('weak') strategy that makes few assumptions – this will have the broadest applicability. Then, have a domain-specific ('strong') strategy that provides more power in a particular setting by leveraging features of the domain.
  Also related is Phil Agre's idea of 'turf', which I think is deeply insightful. When you frame you research and when you share your research, you should try to create and capture as much 'turf' as possible (...but not any more than that because reviewers will penalize you for claiming more than you can support). As Agre points out, creating turf sets the stage for follow-on research and set up a paradigm for further normal science.
• Read and follow Herb Clark's: Everyone Can Write Better (and you are no exception). ...and Bill Griswold's suggestions on reading a research paper are also a great guide for writing one.
• Use standard sections.Describe experiments using APA style: e.g., have sections marked METHOD, RESULTS, etc. Consistency helps readers.
  • Titles: At least at the beginning, title your paper with its contribution. For example, Trust Breaks Down in Electronic Contexts but Can Be Repaired by Initial Face-to-Face contact. I've never failed to remember the contribution of that paper. A counterweight to this advice: all things equal, shorter titles are better. If the paper introduces a system, the title in the camera-ready (but not for review) should begin with the system name, followed by a colon. Why not for review? Omitting the system name during review reminds both you and the readers that you're making a research contribution, not 'merely' building a system.
  • Related Work: A good related work section tells a story. So present related work in that way. Present it synthetically, as what the world currently knows and what holes remain. A bad related work section reads like a laundry list, simply to nod to one's colleagues as "X did A and Y did B and Z did C." Or, "Recent work has also been done on…" Neither of these build a case.How do you write a good related work and not a bad one?
    • Be explicit about what the world doesn't know that you have/will figure out. Usually it's best to integrate the related work into the intro. Because if you can summarize what's been done and why your work is different and better, you've got a paper!
    • For me, the related work section is the one that I most need to write a clear outline for first. You're doing a lot of synthesis and distilling for the reader. So it's hard to just start typing. That leads to verbal diarrhea.
    • Imagine you're having lunch with a friendly colleague who doesn't really know the area. You're about to explain your research, and they've asked you to set the stage. What's out there already? They're technically smart, but they may not know system names / jargon. If systems and jargon matter, you'll need to (briefly) explain them. Which is a good reason to distill and focus on the key issues, rather than name-check a laundry list.
    • Make sure that the story you tell in related work lines up with what the paper delivers on. So if you're motivation is "email overload", then your paper should very clearly reduce email overload. When the motivation doesn't line up with the delivery, that's a bait-and-switch. Which lands your paper in the reject pile.
    • Pro tip: In my experience, it's much easier to write an impact-oriented summary of related work by integrating related work into the INTRO section. Somehow, when it's a section on its own, laundry list problems creep in more.
  • Graphs & Tables: Show your main results visually (i.e., graphs generally beat tables). If you're inclined to make a table, think about whether the reader will benefit from a visual representation. Often, the simplest way to do this is with a bar graph. If people can jump 20 feet with your shoes, but only 10 with others, a simple bar graph will pop out to the reader much more than a table of numbers. Whenever possible, embed sparklines or other iconic data graphics into tables. Visual presentation facilitates rapid comprehension and comparison. Maximize the data-to-ink ratio. When you have bar charts showing an average, augment the bars with whiskers showing the standard error. If you do have a table, avoid (or at least minimize) gridlines, it's just ink pollution. If rules are absolutely necessary, make them 1/4 point.
  • Figures: If possible, put a summary figure on the top-right of page 1. Except on page 1, figures should go on the bottom of the page. Bottom-aligned figures are preferable to top-aligned figures because there is less interference/confusion between the caption text and the body text (because the figure itself serves as a separator). For content in a bitmap image, like a jpg or png, render it at ~300dpi. Use Word's "more layout options" to center figures on the column and bottom-align to the margin. Set the figure to resize to match content, with no internal margins. Make sure there's no leading above the top line or below the bottom line.
  • Figure Captions and Slide Titles should summarize what the reader/listener should take away from the figure/slide. What's the punchline they should remember? Bad: "relationship between median income and beignet consumption." Better: "beignet consumption strongly predicts median income." Left-align captions; easier to read than centered or justified.
  • References: Render references with real reference management software, like EndNote or Mendeley. Don't use Word's built-in system. Why? In addition to automatically consistent formatting and ordering, EndNote/Mendeley will check whether a reference is actually used, so there are no reference orphans. And at some point, you'll need to change how the refs are formatted (what information is included, what's in italics, whether and how to abbreviate with et al., ...). It's faster and more robust for a computer to do this.
  • Appendices: If possible, include your experimental materials (e.g., instructions to participants) in an appendix. If there isn't space, post them on your project Web page.
  • Different field have different customs for reporting p values. To achieve consistency within the group, ours are: use less than with the nearest upper unit (p<0.05, p<0.01, p<0.005,p<0.001) rather than listing the exact value; results <0.08 are a ‘trend’; anything above 0.08 is not significant. For non-signifcant results, report the actual p value, e.g., p=.12. Report other numbers to 3 significant digits: e.g., 45.5, not 45.46.
  • Avoid 'we':Computer scientists use 'we' to mean several very different things, and this causes a great deal of confusion. Sometimes, it marks the authors' opinion or activity (We believe the cronut to be a seminal invention). Sometimes, it's used in place of 'one' (In figure 3, we see a sugar-coated cronut). Sometimes it's used to describe the actions of an algorithm/system (With cronut sort, we first take the oven-fresh cronuts and place them on the top shelf.) You can avoid this ambiguity by restricting 'we' to mean 'the authors'. The middle example becomes: Figure 3 shows a sugar-coated cronut. The algorithm/system example becomes: Cronut-sort first takes... Furthermore, avoid using "our algorithm/system/study" because it's the 'our' isn't scientifically relevant. Hopefully it would work the same if someone else made it. (The 'our' does explain to your friends why you haven't gone surfing with them in two months, but that's a different story.) Replace "our algorithm/system/study" with its name.
  • Number pages on the bottom, starting on page 2. For all drafts, the bottom of Page 1 should say "Draft: Do Not Circulate" along with the date of the most recent edit. Keep the same filename for all draft PDFs. Use the edit date to distinguish versions.
• The final pdf: It's imperative that the archival version of our papers is clearly written and error-free. To insure this, please provide me with a version that you believe to be absolutely 100 percent done. I will make a final pass with a fine-toothed comb. Given that I will need to find a block of large block of free time for this careful pass, expect that it will take at least a week – longer if I have travel or a major deadline. To the best of my memory, every "final" paper I've been given had at least one typo. And every edit has a chance of introducing a new bug. That's why it's essential for me to read a version that you are 100 percent done editing – every figure, reference, data is fully complete. Your final pass should include this careful review of acks and refs:
  • Go through every single cross-reference in the paper (e.g., [45]), make sure it references the citation you intend it to, and make sure the citation supports the referenced statement. References often get scrambled.
  • Make sure all references are bug-free (no typos, garbage, errors like "and et al.") and that venues are cited in a consistent fashion.
  • Acknowledge colleagues who helped you and gave you good feedback, and also sponsors (e.g., the NSF) who supported the work.
• Editor: For the most part, text composition and document layout are separate tasks. Separating these phases will save you time, increase your flexibility, and minimize wasted effort of fidgeting with layout or carefully placing something that may change later.
  • Draft and share your papers with Google Docs so that my edits address the latest version without blocking your progress. That way you can continue to work, and when I read your paper, I will see the latest version. Google Docs are ideal for this because if I get halfway through, you'll see those comments immediately. Neil Patel really pioneered this in our group.
  • Once the paper is mostly done, move to an editor that supports layout. Hopefully, a web-based tool like Google Docs will eventually be able to do this. Right now, our two main choices for a layout editor are Microsoft Word (shared via OneDrive) and Latex. Both have major idiosyncrasies, and both have major drawbacks. (My sense is that other options like Apple's Pages aren't quite ready for prime time – for example, Pages only works on Mac. InDesign supports fancy layout, but is overkill for our purposes and limits collaboration since few have it.) Word's biggest benefits: WYSIWYG editing AND commenting; track changes; interactive spelling and grammar feedback. Word's biggest idiosyncrasy: figure placement (it's both flexible and deterministic once you know how). Word's biggest drawback: good reference management requires separate software (like EndNote or Mendeley). Latex's biggest benefits: text-only representation integrates well with source control; great for math; bibtex. Latex's biggest drawbacks: time wasted compiling and rendering documents; reading is slowed by monospace editors littered with formatting syntax; commenting/revisions are clunkier; layout is less flexible. (don't believe me? One quick example: send me a latex paper with a figure that spans 1.5 columns.) I have written papers with many dozen people, and this experience has taught me that Word's drawbacks waste less time than Latex's. (Sometimes, it does feel like a race to the bottom.) So, unless you have a math-heavy paper, please use Word so that we can collaborate most efficiently. I want to emphasize that this is a data-driven choice rather than an ideological one, so if new data or options appear, I'm happy to revisit.