Working with Scott’s Group

Communicate

• Each week, share a weekly update with crew comprising: what you accomplished that week, what you plan to accomplish in the coming week, and what your updated safe and stretch goals for the quarter are. While it's fine (wonderful even) to include a line about coursework or attending a talk, keep the bulk of your update focused on lab/research endeavors. Consider sharing your 'let's do this' gSheet.
• Mailing lists: Ask Ops to add you to locals, & me to add you to crew.
• The Design Lab Google Calendar lists the speakers for research charettes, Design at Large seminars, conference dates and deadlines, and other design-related talks and events.
• Speak at Weds lab meetings 2-3x a year to get feedback on early-stage ideas. Earlier is better than later. Lab meetings should primarily be discussion time, not presentation time.
  • Whether a really early brainstorm, a mid-project check-in, or a practice talk for completed work, begin by sharing three things with the audience:
    1. What's your research goal? It's hard for the audience to suggest directions unless we know where you want to go.
    2. What's your point-of-view/ hypothesis?
    3. What kind of feedback do you (and don't you) want from us?
  • Often, you'll get three slides in, and a discussion will erupt. So… front-load your take-home message. If you have to repeatedly say, "I'll get to that later in the talk," the order is wrong. Lead with your theory, results, and punchline. In research talks as in journalism, a pyramid presentation ensures that even if one doesn't get all the way through, the key take-home points are up front. Yes, figuring out how to sequence a talk is really hard – everything is interdependent, so what goes first? And you've done far more work than you can share in 20 minutes (about the amount of presentation time you'll get in a highly-interactive discussion) – what's in and what's out? All the more reason to diligently practice talk sequencing and selection as often as you can.
  The role of non-presenters is to intellectually engage with the ideas and ask questions, like:
  • what assumptions might be unfounded? What opportunities does that unearth?
  • Does the question you asked line up with the research program and the results? When they don't line up, how might that inform work going forward?
  • What do you seek to accomplish? What does success look like? Why is research needed (i.e., where specifically does prior work come up short)?
  You should expect that your lab-mates are your harshest critics. (And also your greatest supporters.) This careful scrutiny helps achieve two important goals. First, we take our role of knowledge-building very seriously, and this lab has impressively talented students. Often that means holding ourselves to higher standards than others do, so we'll ask harder and more challenging questions than other places might. You shouldn't expect that all of them will be answerable right away. Second, if your hardest sparring partner is in the safety and privacy of a lab, that makes conference talks, job talks, etc. a walk in the park.
• Use headphones & a wired Ethernet connection for all video meetings.

• Email. If you have a question for me or would like to share an update, send me an email. Give it a specific, action-oriented title. My average response latency is inversely proportional to email length. Brevity is divine: follow the email charter.
• In person. Drop by my office if you have a question; late afternoon is best. I try to keep mornings interruption-free for longer, focused work. (Undergrad RAs: office hours are the best time for non-research questions like job advice.)
• Complex, thorny questions are best handled in person, rather than email. They often require discussion to work through, and somehow I often find that in-person conversations are just better stuited to complex topics than typing email. (Also, complex emails get buried.) Shoot me an email ahead of time, so that I can noodle on it for a few days. Say "here's a heads-up that I'll have a question for you at our meeting". That'll give me more incubation time. I will read your email and reply in person.
• If we discuss something, write it down. It wastes valuable time to have to come back and say, "I forgot to write down what you told me, can you say it again." If we agree that I will do something, email me a short reminder that I can use as a starting point. The crisper the ask, the sooner I'll complete it.

• When you schedule a meeting/event (e.g., if I or others agree to meet with you about research) send all attendees a meeting request through Google Calendar or iCal with the time and location so that is shows up in everyone's calendar.
• The evening before/morning of the meeting, send all attendees a reminder email that includes a one-sentence agenda. Feel free to send a longer agenda. The key is to send at least something.

• Hang posters that you present in the Design Lab or Hallway outside the lab. It breathes life into the space.
• Have a Web page for your project, and keep it up to date with papers, video, code, news, pictures, etc.
  See the ButterflyNet and d.tools pages for examples.
• Use a ucsd.edu (or socsci./eng.) email for all internal and external research communcation, shared documents, and repositories. I know who you are, but others may not. Email domain is the letterhead of the digital age. (It's fine to use gmail as the 'interface' to your email as long as you configure the from and reply-to to a ucsd address.)
• Submitting a work-in-progress, demo, or poster to CHI/UIST/CSCW/L@S is a great way to share in-progress work with the world and get feedback. These are not 'archival' venues – they're an invitation to discussion. (They don't go on the 'papers' section of your CV and they don't preclude or inhibit future publication in archival form.) Our group sees three main benefits to your sharing work through these venues:
  1. You get practice with the publishing cycle: writing, revision, submitting, peer review, and presentation. That way, when it comes time to publish a 'real' paper, you will have been through the cycle already.
  2. It gives you a chance to see how the world reacts to your work, offering you the opportunity for course correction. It's valuable to know sooner rather than later if someone is skeptical of your result, believes you're missing a key piece of related work, sees an opportunity you missed, or understands things better presented one way than another.
  3. As Ranjitha writes, it's a "great end-of-rotation deliverable. Often students are working in the context of a larger research project and are rushed to complete research by the end of the quarter... having a written deliverable really helps them understand what their work meant in the scope of a larger research direction. It also creates a sense of ownership over that work, so that it isn't something that they do for a quarter and never think about again."
• Demo your work regularly — it’s a great way to get feedback and users. Remember, a demo is a story, not a functionality test. Start by explaining why someone would want to use your system, and walk them through a scenario.

Be present

The corollary to this: make sure to get out of the lab :) To make your time in-lab focused, you'll need time out of the lab. Also, it's California! There's all sorts of wonderful natural and cultural things out there…

Attend three regular gatherings. At all three, listen actively and ask questions. To be fully engaged, put away your email devices and disable interruptions. Sit at the central table, not on the perimeter. If the central table seems full, ask people to make room.

1. Lab Meeting: These are informal, discussion-driven talks where you should regularly ask questions. (And please help clean up after.) Sign up to present at the end of your rotation/internship. PhD students should expect to present 2-3 times a year. Sometimes it will be a practice talk for a formal presentation, like a conference talk. Other times, it will be a chance to share early thoughts on a new project. If you're the speaker, ask a friend to take notes for you.
2. Design at Large Seminar (Academic year). Attend the talks & sign up to meet with visiting speakers. We get world-class speakers and this is your chance to ask them questions. This is also your chance to offer a 'calling card' about your research. Officially register for cogs229/cse219 – it's 1 unit and helps convey student enthusiasm for bringing in speakers.

   When you introduce yourself to a visiting speaker, say something provocative, not administrative. DON'T SAY, "I’m working with Scott on the donut project." a) that’s boring. b) they don't know what the donut project is. DO SAY, "I'm investigating whether we can achieve a new level of donut tastiness by cooking them in sous vide." "I'm investigating" is one good lead. "I'd like to find out" is another.

3. Project meeting
   • We'll meet in the afternoons, three times every two weeks. One week on Monday & Friday; the next on Weds. It sounds a little strange, but works great.
   • At our scheduled meeting time, please come in and set up. Do not wait outside (even if someone else is in my office, or I haven’t arrived yet). Plug your laptop into the shared display. Open up your to-do list for the week, any artifacts you would like to share/discuss, and a place to take notes. Prior to our meeting, close all extraneous browser tabs and disable all interruptions, like IM. Email/chat notifications should not pop up during our meeting. You can connect to the Apple TV to wirelessly share your screen.
   • Begin presenting your agenda.
   • Take notes about what we discuss and how you will follow up. Right after meeting, email me the summary of decisions and your revised plan for the coming week.
     • Notes serve as an important reminder for you. (Distribute cognition!) Also, taking notes conveys to all present that what we discuss matters.
     • Take notes in the same place each week, and keep your notes together. Keep all of your notes in one plae: avoid loose sheets of paper, scattered text files on your hard drive, or meeting notes interspersed with random other things.
     • I use Evernote. Many use Google Docs. Others like paper. Digital has the advantage of search. Paper has the advantage that it makes clear to all what you’re doing… and it won’t buzz or tempt you to check email. Use anything you like, as long as you’re consistent and keep things in one place.
   • For collaborative projects, keep a Google Sheet of tasks with estimated time and due date. Google Sheets are easy for multiple people to edit simultaneously. Write tasks to capture the concrete next action. Avoid vague tasks because they’re harder to start. If needed, split a vague task into multiple concrete tasks. Show this spreadsheet when we meet.
   • Having a regular meeting schedule helps keep me sane. I love research and working with students. I hate spending time on coordination emails. Work hard to make our meeting, so we don’t have to reschedule. That said, there will be times when our scheduled meeting won’t work, e.g., one of us is out of town. In those cases, please let me know well in advance (e.g., when you book your travel). My calendar can keep track of things far into the future, and so knowing a month in advance that we won’t meet on a certain day provides me with valuable flexibility in case something else comes up.
   • If you need me to do a quick task like sign a form or send a short email: bring a filled-out form (or drafted email) to our meeting (or office hours) and wait while I do it. Two benefits: 1) I'll do it right then; 2) If I have questions or need to be reminded about something, I can ask. When possible, please avoid email for such things.

Your first day

1. Set up your laptop at your desk. We can provide a Macbook, monitor, keyboard and/or mouse for your time in the group. Email me a list of equipment you borrow so we can track it.
2. At your desk, read this page and its linked documents. I estimate it'll take you an hour-ish to read and contemplate, and another 2-3 hours to skim the links (but not the mentioned books). By the end of your first day, email me with anything that you may not understand or agree with. I assume that set is non-empty :). That's important so that we can have common ground in our collaboration.
   My students suggest re-reading this page after 6 months or so. It's a lot to absorb at once. Also, the relevance of some things may be clearer after some time in the lab.
3. Put your name on/above your desk: hand-drawn, Spell with Flickr, ...the world is your oyster.
4. Take the UCSD human subjects course and email me your completion certificate. You must be certified and your study must have IRB approval before running any participants in a scientific study. If any author of a paper lists UCSD affiliation, then you need UCSD IRB clearance, even if the study is executed by collaborators elsewhere.
5. Our group is firmly committed to treating each other and everyone we work with with respect. One benefit of being at a university is there are many resources for helping us learn to do so, and for addressing issues when they arise. One resource is the Office for the Prevention of Harassment and Discrimination. Please complete their short training course and send me the completion email you receive.

Organize all research files (design work, code, writing, presentations, data).

• Store all research files in one of four places, and nowhere else
  Do not keep copies of research/teaching files e.g., loose on your computer's desktop because they risk getting lost.
  • Google Drive · Put all your data files, plans, outlines, papers, proposals, & talks in the Research folder. Many benefits to consistently putting work here. An example: it syncs to my (and your) laptop, so I (and you) can get to your paper more easily. The DLab gDocsfolder has information about the lab.
    I strongly recommend using the Google Drive OS X app
  • When multiple people are actively editing papers (i.e., the week before a deadline), put the current draft on OneDrive. When the editing flurry ends, move papers back to Google Drive.
  • The d Web server · Web content
    From the Finder, connect to smb://d.ucsd.edu. If you don’t have access, email sscf-cogsci@ucsd.edu. Database-backed information, like publications, lives in the MySQL database.
  • The DesignAtLarge github / svn (code)
  Use a ucsd.edu email address for your account on these services.
• We have a Time Capsule device for backup. It's also helpful in case you change devices and want all your software move easily from old to new macbook.
• Clearly name and organize your files so your collaborators can find them without having to ask you. For all of your research (study data, proposals, publications, talks, etc.) name folders ProjectName-VenueYear and files ProjectName-VenueYear-Version, e.g., dtools-UIST2006-17, or Bricolage-NSFIIS2010-23. Keep all drafts in subfolder named drafts, so that the only file in the main folder is the current version. DO NOT use generic filenames / titles like "my_chi_paper.doc", "paper.pdf", "Experiment Design", "talks", or "research." This holds for Google Docs too.
• There are three reasons why it's essential for you to be very diligent about keeping all of your files consistently organized in a common, shared repository.
  1. if your computer is lost, damaged, or stolen — or your hard drive crashes — no research should be lost.
  2. shared repositories minimize coordination problems when collaborating, and provide versioning support.
  3. it helps me and other members of the group find your work when you’re unavailable (or have graduated!).
  As the group has grown, different people have gravitated toward different data management strategies. And I empathize with that. But, when different people or sub-groups have different strategies, that makes it much more difficult for me to find information and for sub-groups to collaborate with each other. I don’t really care what repository we use. The key criteria is that whatever we use, we all use it consistently, and that it works for people’s wide-ranging needs.
• Mark all your code as BSD Open Source. Clearly name all your variables, so that others can more easily understand your code.
• Show Me Your Data: Please share data early and often. Organize your spreadsheet so that the main dependent variable is a column, with its entries in the rows. Title each column clearly in the first row. Anyone in the group should be able to interpret them.
• I strongly recommend the one-computer strategy (and suggest a Macbook). You’ll spend less time keeping your machine up-to-date. And you’ll never have the problem that you can’t do a task because a file/ application/ password/ configuration/ setting is on your other machine. Why a Macbook? It’s what everyone else in the group has, and consistency helps collaboration.
• If you have paperwork questions or need a signature, send it to srk-admin@ucsd.edu. Requests to srk@ucsd can get buried.

• Here are some examples of research posters from our group.

I expect you to…

• For important events, you should be available to demo your work. I’ll shield you from lots of these requests so you don’t end up sliced too thin. The flip side is that when I do ask, you can be sure it’s important. We're fortunate to have a strong group of supporters; the least we can do is show them what we’ve done with the resources they’ve provided us.
• To keep the group running smoothly, there are several support roles that graduate students play, such as coordinating the research meeting schedule for a quarter. Each year, you should expect to play one of these roles.
• Focused: Dedicate time each day for serious research, and use that time to make progress toward the research goals you outline in your weekly plan. During this time, limit distractions: turn off your email/chat/… so you won't be interrupted, and close all applications/documents/tabs that are unrelated to that day's research. If you have 18 applications open, 27 random tabs open, and six things bouncing and blinking on your screen, that's not an environment conducive to the focused, reflective work that is essential to performing deep research. Of course, not all your time needs to be (or even can be) deep and focused, but some of your time should be.
  • In general, the best predictor of performance in any field is perserverence and time on task. You may have heard Anders Ericsson's oft-publicized estimate that it takes 10,000 hours to become an expert in a domain. That's obviously an oversimplification. But the high-level takeaway is spot on: expertise requires many hours of focused practice. And – even at the highest levels of achievement – time on task is the best predictor for differentiating the really good from the pretty good. Malcolm Gladwell summarizes Anders Ericsson's research on this wonderfully in Outliers. Corollary: high-quality time is essential. If you're chronically sleep-deprived, you can't have high-quality time.
  • If you're in the research group, you're not at a startup (and vice versa). As I write on my public FAQ page, both are (much more than) full-time endeavors. Moonlighting just means both endeavors lose.
  • Industry internships are valuable to extend your network of collaborators and gain a different perspective on your work. It's totally worth doing one internship during your PhD (and probably not worth doing more than one). One good strategy: do an internship right after you submit a big paper (doesn't need to be summer); that lets you pipeline.
• If you find yourself thrashing or feeling like you can't stay on top of your tasks: the most useful advice I've found on how to create and organized tasks is David Allen's Getting Things Done. I'm also a big fan of Paul Graham's Maker's Schedule, Manager's Schedule.
• Apply for funding: Our group takes a somewhat socialist approach to supporting PhD students. If you are a PhD student in the group working hard towards one of the group's major research goals, everyone in the group will work hard to fund your work rain or shine. This means that in general:
  • when the group collectively brings in sufficient funding, that collective achievement will support your work. (provided, of course, that it contributes to a major thrust of the group and you are working really hard and making serious progress on research. We don't have a mechanism to support 'random', unrelated research ideas. And, given the value of cumulative research, that generally wouldn't warrant support even if we could. It's also not a good use of the group's grants to support students that aren't working really hard: if you aren't serious there are a lot of people that are. And we can't sustain funding unless we can demonstrate amazing results.)
  • Your contribution to this process is a) to help write grants and apply for student fellowships, and b) provide your peers with feedback during lab meetings and to drafts they send to srk-students.
  • Over the course of your PhD, you should expect to be a lead (co-)author on 1-2 'substantial' proposals (e.g., like an NSF grant that will support 1-2 students for 3-4 years) and a couple smaller proposals (e.g., a private-sector opportunity at the scale of a couple pages of writing for ~1 student year of support). Over the course of a 5-year PhD, expect grant-writing and maintence (annual reports, talks, etc.) to add up to 2 months of work on your part. Big grants will take you 2-3 weeks. Short grants will take you 1-3 days. When you take the lead, you are responsible for making sure all of the pieces are submitted properly and on time.
  • In addition to providing funding, grantwriting is a great venue to force you to clarify and flesh out your ideas; most students find this an intrinsically valuable activity. And your grant/fellowship proposal can also be your thesis proposal – kill two birds with one stone.
  • Funding is best thought of as an expected-value activity. Let's say our hit rate is ~50%. That means we'll need to apply for double the grants we need to fund the group. It also means that any given person on any given year may not hit paydirt. And on another year, one may hit paydirt multiple times. The process's intrinsic randomness is a major reason we operate on a quasi-socialist model. For example, as a first-year student, you may be covered on a grant written by a senior student and I. Then, you may write a proposal that doesn't get funded. Later on, the department may successfully nominate you for a fellowship. Regardless of if/when/how many of these things work out, each of us contributes on the grantwriting side. You get the idea. We all work to bring in funding. And because of the randomness, we all share the rewards.
  • The group will work to support you on a consistent, long-term trajectory so that your dissertation research can build cumulatively. That said, it may need to bend a bit to accomodate current funding constraints. For example, if the group has funding on mobile research, it will be important for your research to have a connection to mobile.
  • The best way to write a grant is to write down the research that you would like to do. Think of proposals first and foremost as a research plan, as if you were writing to a colleague. Use it as an opportunity to brainstorm and flesh out future projects, and read and understand related work. The 'spin' of grantwriting should absolutely play second fiddle and happen only after you've figured out your research goals.
  • With grant-writing, it's especially important to begin with a clear and detailed outline.
• As an incoming or first-year student, apply for fellowships you are eligible for, like the NSF Graduate Fellowship.
• If you are a senior PhD student, apply for dissertation fellowships and things like the Google, Microsoft, and Qualcomm fellowships.
• Proposal writing is one of the world's most difficult tasks because it requires you to envision and project a clear, coherent, exciting vision of the future. To manage this difficulty, I suggest a four-step process for writing a fellowship application
1. At least a month before the deadline, select your letter-writers and contact them. If I'm one of your letter-writers, follow these instructions. In your request email, offer to write a draft for the letter-writer. If they accept...
2. send your letter-writer a draft letter for them to work from. This can be daunting because it can feel like bragging. Ask me for an example letter that you can use as a template.
3. At least two weeks before the deadline, bring a detailed outline to our meeting for discussion. Remind me to expect this, and put it in your weekly plan, so I can help you stay on task.
4. Turn the outline into a proposal :)
• UCSD lists its internal deadlines here.
• As a general policy, we don't sign NDAs. We ask companies not to share with us anything that they feel is confidential for a very simple reason: if we don’t know it, then nothing we do or say can be tainted by it. If anyone asks you to sign an NDA, please decline, citing our lab's policy and emphasizing that it's in all of our best interest for companies to not share confidential information with us. If something mission-critical requires you to sign an NDA, let's discuss it.

Write clearly

Speak Effectively by being prepared.

Conference Travel

To design and analyze experiments effectively

What qualifies someone to be a co-author on a paper in our group?