How to Make Remote Work Effective for Data Science Teams

In 1973, at the height of the OPEC oil crisis and skyrocketing fuel prices, NASA scientist and USC professor Jack Nilles began thinking about ways work could be done without the need for commuting. Nilles’ thought experiments evolved into case studies, numerous books, including The Telecommunications-Transportation Tradeoff, the original book on telecommuting, as well as dozens of papers, articles and keynote speeches. To this day, Nilles remains one of the principal evangelists for remote work as a viable alternative to a traditional office.

Organizational evolution of telecommuting at two different companies, from The Telecommunications Transportation Tradeoff (Nilles).

Today, it would not be entirely hyperbolic to consider Nilles’ early work clairvoyant. The last decade has seen a dramatic rise in companies adopting a remote workforce. A 2019 report compiled by Flexjobs and Global Workplace Analytics found that the number of people telecommuting in the United States increased by 159% from 2005 to 2017. With the rapidly evolving global crisis surrounding COVID-19, forcing more and more companies to adopt strict work-from-home policies, remote work is further cementing itself as a workplace paradigm to be taken seriously.

The data science industry is no exception. Large computer, IT and software development industries are among the fastest-growing with respect to the share of employees working on a full or partially-remote basis. The perks of remote work are well-documented: a joint report published by Buffer and Angellist — a survey of more than 3,500 remote tech workers — found that a flexible schedule, the ability to work from anywhere, and not having a commute were among the most attractive benefits of remote work.

Buffer and Angellist Report

Despite these benefits, putting the systems in place to make remote teams effective and productive is not trivial. Simple things like collaboration and communication become challenging. Employees are more prone to feel lonely, distracted, or to feel like the workday never ends. In data science teams, where industry best practices are still very much taking shape, these issues can be even more pronounced.

Buffer and Angellist Report

This post aims to highlight some work from home best practices, both general and data science-specific, in order to help data scientists and teams remain productive, connected and happy while working remotely.

Best Practices for Members of Data Science Teams

Create a Productive Workspace

Have a dedicated workspace (read: not your bed). Keep your phone away from your workspace and only check it during breaks. Be sure to keep your workspace clean — working in a messy environment can be incredibly distracting.

Make sure to chat with your colleagues — even when you don’t have to

Schedule a couple of video calls a day to stay sane, even if they’re not strictly necessary. Have at least one of these be a non-work related ‘remote coffee’ with a colleague/friend. Always put the camera on and try to enlarge the video call window so it really feels like you’re having a real conversation.

Working habits

Block time in your calendar for undistracted work time, a workout, lunch, etc. Encourage your colleagues to do the same. Over-communicate and assume good intent. A quick update message can go a long way in ensuring your manager is reassured with the work’s progress. Relatedly, agree on a daily cadence to stay in touch with your teammates — e.g. a daily standup meeting each morning.

Have a clear ‘turn-off time’ (agreed with your team) — one of the biggest risks of remote is that you never switch off and stop working so better to be proactive on this one

Tips for Managing Data Science Teams Remotely

When thinking about how to manage a remote team it’s important to start by highlighting the key benefits and downsides of remote work.

At a high-level, the benefit to you as a manager is the potential of increased productivity for your team due to no commute, fewer distractions (hopefully), and more time to focus.

The downsides generally are less effective communication and fewer opportunities for collaboration. Some may argue you can be just as collaborative and communicative remotely, but as any economist would tell you: there is no free lunch. While remote teams do come with downsides, there are some very good strategies by which you can minimize them.

Take Advantage of the Pros of Remote Work

First, focus on and enable the strengths of remote work. You have just been given a team with the capacity to focus even more than before. Use that! In order to do so, you will need to do the following:

1. Trust your team. If you take one thing away from how to manage a remote team well, let it be this: trust your team to work effectively and manage their time well. If you spend a lot of time doing more micromanaging you will give up almost all the benefits of remote work.
2. Set clear expectations. If you can set clear work priorities with measurable milestones, you can let your team go to work and focus. At this point, working from home can be a huge advantage because of hopefully fewer distractions and more time to get work done.

If you can set clear expectations and trust your team to get the work done, working remotely can be an advantage. For example, you could check-in at the beginning of the day with the team, set expectations, and then let everyone go to work.

Build Systems and Processes to Counterbalance the Downsides

Now — let’s discuss the downsides. Communication and collaboration. Focused work is great when you know what needs to be done and can accomplish it mostly on your own. But what about those serendipitous moments of in-person collaboration? How do you replicate that? I would argue that you can’t completely. Communication and collaboration are probably always going to be better in person (thus, no free lunch), but that doesn’t mean you can’t minimize the downsides of remote work. To do so, I would recommend the following:

1. Communicate over video. A lot of how we communicate is non-verbal and not turning on your video during a call hurts your ability to communicate.
2. Don’t always go straight to business in your meetings. With a remote team, you don’t get time to have “water cooler” discussions. Spend some time in meetings, when appropriate, just talking to stay connected.
3. Have scheduled, remote-friendly team building activities. The “water cooler” discussions should happen every day, but I would also suggest a monthly activity with concentrated time to relax and get to know each other. One of my team’s favorites is Mafia.
4. When the next steps are not clear and you need creative collaboration, do remote brainstorming using the whiteboard feature on video conferencing platforms. Don’t just get in a virtual meeting and start discussing ideas. Send out the goal in writing beforehand, have people brainstorm individually and bring ideas to the meeting, and allow everyone to share his or her ideas, then discuss and prioritize. This is sometimes called Silent Brainstorming and is very remote-friendly and tends to be more effective.
5. Develop a culture of radical candor and psychological safety. One of your weaknesses is now communication because you are remote. Make sure your culture is especially strong in being open and honest with feedback. Both positive and negative! As a leader, be an example in asking for, giving, and receiving feedback. And make sure everyone feels safe in expressing ideas and giving opinions.

These steps should help build camaraderie and trust within your remote team, both of which are necessary for strong communication and collaboration. You don’t have the luxury of falling back on serendipitous moments of in-person collaboration or the post-meeting discussions that happen when people are in the same room.

Invest in Tools to Encourage Collaboration

There are amazing tools that you can leverage to make you even more effective with your communication and collaboration.

Zoom

A good video conferencing tool is indispensable for all teams, especially teams that work remotely. We use zoom every day for meetings with remote team members. Whatever video conferencing tool you choose, be sure it’s easily accessible so your team can have as little friction as possible getting on the phone with colleagues.

Slack

Real-time messaging allows the flow of information across a team to be ad-hoc, conversational and informal. These are crucial values to ensure you maintain the ability of your team to develop ideas easily, organically, and collaboratively. Slack is a popular and excellent tool that allows teams to do just this.

Comet

Data Science teams are prone to becoming packs of lone-wolf modelers even when working in the same place. With the switch to remote, it’s even easier to fall into this trap. Fortunately, tools exist that make it far easier for data scientists, machine learning engineers, or anyone working on models to log, visualize, share and reproduce their work across distributed teams in real-time. Comet provides a meta-machine learning platform, runnable in the cloud or on-premise/VPC, that allows data science teams to do just this: reproduce full experiments (and not just code), manage users across large distributed data science teams, and provide managers insight into team contributions and performance.

Comet allows data science teams to easily manage distributed teams by creating collaborative projects with many users:

Adding Collaborators in Comet

If your team is part of a collaborative project, every data scientist can publish their experiments in one central location, where visualizing, comparing, analyzing and debugging team-wide work becomes easy to do — even when working remotely:

Comet Project Page

Conclusion

With COVID-19 forcing many teams into remote work, members and managers of newly-remote teams will be well-served taking stock of how to make the most of their new telecommute. Teams that take advantage of remote work’s benefits will remain productive, communicative and happy. What’s more, safeguards and processes can be put in place to effectively counterbalance many of the common challenges remote teams face. Do yourself a favor and schedule a 15-minute video call with one of your coworkers later today. It will be time well-spent.

This article was co-published with Nikolas Laskaris of Comet.ml. I am a user of Comet and wasn’t paid by them for this article.

Interested in learning more about Python data analysis and visualization? Check out my course.