May 3, 2023 PAO-05-23-CL-05
Casey Jenkins (CJ): epocrates is a clinical guidance tool that is used primarily in the United States. We have reached over a million healthcare practitioners — nurses, physicians, pharmacists, and so on — with content that we’ve developed or that has been distilled by our team of authors and editors into sort of concise and pithy bits of information. The epocrates app began as a consolidation and summarization of drug monographs, and it has been primarily used as a drug reference tool since its inception. It has expanded that over time to include things like guideline reviews, disease state information, and other tools like calculators and lab reports.
epocrates is primarily used at the point of care because and that is when physicians most need easy access to that information. Our primary focus on drug tools makes us unique; other tools in our space are more social in nature or focused on diseases. epocrates remains far and away the primary place to go for prescribing information: dosing, interaction checking, and so forth.
CJ: epocrates has certainly come a long way over the course of the last 25 years. It originated in the late 1990s as essentially a handwritten summary of drug monograph information. It was one of the first medical information tools — if not the first — that moved online, originally as a website back in the early days of the Internet.
From there, we moved into something called a “DocAlert.” There’s a lot of alerts in this space, but we identified a clear need to collect medical, drug-related news –– a new drug launch or new formulation or dosing or something of that nature –– and to develop clear, direct news stories covering that information and publish them on our channel to reach practicing clinicians.
Over time, that evolved into advertising, around the time that the web was getting up to speed and starting to move into the new app environment. During one of the first previews of the app store itself, epocrates was one of only four apps invited to present on stage by Steve Jobs. We have truly been part of the mobile information look-up and advertising of space from its inception. Over time, we have continued to add more information and features as they have either become available or our users have expressed interest in their utility.
We didn’t come out of the gate with disease information; we started with drug information and then moved into disease. Around 2005–2006, we added features like calculators and other guideline tools. We have continually evolved and refined both our content offerings and our interface. More recently, we’ve introduced continuing medical education (CME) information into the mix; healthcare practitioners benefit from the unbiased learnings they get from these CME activities and earn important credits, and we saw both the interest and opportunity to provide those through epocrates, since it was a tool that those practitioners were already using and happy with.
We continue to launch new useful tools. Last year, we debuted our “Bugs + Drugs” tool, which is an antimicrobial resistance tracker that allows clinicians to explore the bugs — and the antimicrobial resistance — that are being reported across area locality by zip code. It provides a really nice view to prescribers who are investigating or dealing with a potential bug about what antibiotics and treatment course might be best in their area for that type of issue.
There has been a pretty significant transformation from a relatively static, drug-only tool through the simultaneous evolution of the app movement to ultimately become the primary place where drug and disease information is viewed by our user group.
CJ: Exactly. When it comes to product development and management, this is all the stuff that happens in the background that folks don’t always think about. The content management and web distribution systems that were in use 10–15 years ago were completely different from what exists today. We’ve done backend work to move all of our systems to the cloud like everyone else over the last 5–10 years. We are constantly evaluating the efficiency and the performance of our code and our backend fills and refactoring things with new technology, languages, and systems. Any time you run a digital product, this is the cost of doing business that happens in the background that I think a lot of people take for granted; but what a lot of these teams are doing is constant maintenance and evaluation of the available tools and services. That is all complicated further by all of the security issues and compliance protocols that need to be maintained and enhanced over time, which is typically supported by vendors who specialize in that space.
Back in the PalmPilot days, we were essentially just pushing out lists of the sort of proto-XML file that the Palm Pilot could handle. Today, we have a continuous publishing apparatus where we can sync content across multiple nodes and distribute it across the country. It has been quite an evolution.
CJ: We have had many conversations about the evolution of digital technologies when it comes to data privacy and the privacy concerns specifically in the healthcare sector, and there are many considerations. We’re very careful about what data we collect, how we collect it, how we store it, and what we can and can’t do with it. We can view trends in data that have been appropriately de-identified; the ”Bugs + Drugs” tool is a great example. We can examine these trends in microbes and resistance from the level of zip code or region rather than a specific clinic or physician, which is the best balance to make the best use of the data within the limitations.
We are also able to see seasonality or topicality in the types of drug information that people are looking up. We know when flu season is starting because we see a lot of searches for flu symptoms or for things like Tamiflu. We also see spikes in our raw query data for drugs or diseases that are topically in the news. For example, during the last Winter Olympics, there was a situation where figure skaters were being flagged for using trimetazidine, which was banned for competition — that drug was the number one searched drug that week. We see those kind of trends driven both by external events like news coverage and by the direct experience of our users.
CJ: We have begun exploring deeper levels of personalization, more specifically interest matching. We can do this easily with our new CME offering: after you take a given course, we can suggest another course that would likely interest you. We do the same with many of our alerts, so we aren’t, for example, sending gastrointestinal-centric news to a cardiologist.
CJ: Overall, video has really become ubiquitous in a manner that is agnostic of the channel or the user segment. For epocrates, video content can be anything from a short, attention-grabbing element to convey a single critical piece of information all the way through to long-form educational content; for example, CME videos are usually in the neighborhood of about 45–60 minutes. We’re not producing that long-form of content ourselves; we partner with established education creators to create unbiased and compliant content.
We host video content and drive access on our channel. It’s a very different use case than trying to convey a piece of information to someone. The “Message in Motion” product is generally more of a commercial product or an alert-style product. Where before we would use static text or a traditional banner, we now have the ability to introduce short-form video content into those spaces. We know that motion on apps is effective in catching attention, and there’s much stronger engagement with video-style messaging than with more static content. A number of different research studies have shown that information is typically more reinforced through interactions with video.
CJ: Physicians don’t typically want to watch a 5- to 10-minute video, much less a 30-minute one, when they’re in clinic. Our intention with the message-in-motion content is primarily to drive awareness. We can host those the longer-form videos, and we’re working with our partners to find a place for them within our ecosystem, but the primary way that our physicians are interacting with us remains at that point of care.
It's sort of like a two-step process at present. We want to grab their attention in the moment at the point of care but also point the way toward longer-form content to explore later, on a break or in the evening, for example. We’ve recently seen an uptick in usage of our application in what we typically would consider off-hours, both in terms of engagement itself and the duration of that engagement. So, although we are targeting the same audience with the short- and long-form video content, the contexts of use are generally quite different.
We have video in two locations in our product: within the CME modules and in the newsfeed. They are very different use cases. For CME, it’s about learning, and helping clinicians keep up with the latest research. The length of the module determines the number of credits, so there’s an acceptance of longer videos.
On the other hand, our newsfeed is designed for use before, after, and in between the moments of care. It gives busy clinicians a quick view of the headlines and access to a deeper dive. We recommend shorter videos in this location.
CJ: At present, our partners are producing content of all types. So far, we have found that a mixture of talking head KOL material and graphical elements is effective for longer-form video. For the shorter-form, attention-grabbing videos, more graphical material tends to get a better response than 20 minutes or 20 seconds of someone sort of talking. It all depends on the outcome that we’re trying to achieve on the channel through the deployment of that particular asset.
CJ: I think that it depends on the person. This is a question that we have extensively explored from a product development standpoint, which is one of the reasons why we always try to leverage a multitude of approaches. We have a wealth of usage data that we will leverage as these partner videos go live so that we can develop a better understanding of video interactions.
To be very clear: we keep our editorial group completely separate from our commercial teams; there’s an imaginary wall between those two groups. We are always very careful when we are working to deliver more intrinsically commercial messages that we always do so with the integrity of our brand and of our users in mind. Quite frankly, our partners on that side, the agencies and the brands themselves, are very mindful of that as well, and always produce very high-quality assets.
CJ: That’s a really interesting question, and there are a lot of different considerations in play. In the short term, for user-generated content as it relates to clinical practice, the answer is probably no. There are a couple of organizations exploring that, but it’s very complicated from the regulatory and compliance perspectives and not something that we are interested in doing anytime soon.
However, we are looking at other types of interactions with our users. We have a couple of interactive products already, such as polls that we deploy on channels to engage with the community. We are exploring additional ways to let our users participate beyond just looking up information in our ecosystem; but not to the point of posting actual medical information and trying to create a channel for discussion or advice.
On a very different level, when it comes to feedback about the application itself, such as the utility and the usage of the app or its content or features, we have a very robust mechanism and channel for listening to that type of feedback. Our product development process is driven by tried-and-true methodologies around product management and product development: understanding the market, understanding the need, and working closely with designers and information architects and then going out and iteratively testing and tweaking things to the point where our user test groups are providing positive feedback about the content, its utility, and how things have been designed.
CJ: As a product manager in the digital space who’s been working in digital and reference content for 20 years, that’s something I think about all the time, and I have explored this with organizations and partners over the years. The changes that I’ve seen in the technology and the capabilities of machines to help in those processes have been mindblowing in many ways.
I especially think what we have seen released publicly in the last six months or so with ChatGPT and the Google versions and the Microsoft versions has blown anything I had ever seen before out of the water. I have worked with some partners to see, for example, if AI and language models can take the thousands of research papers that are being published on a regular basis and synthesize things down to a format that a human could realistically work with and curate and edit. Historically, AI was just okay at tasks like that; even as recently as two years ago, I ran some experiments, and the output was just okay — not great. But what I’ve seen recently has been somewhat astounding. I think we’re still assessing what that means, but I can see a place where the synthesis of news information or of a research study could be put through one of these systems and then given to a human for review and ultimately be used in publication. I still think that moving directly from the machine to clinical publication is a long way off and may never come to fruition, as there is simply no tolerance for error or ambiguity.
CJ: We are constantly trying to better understand the market and our users’ needs. Video will play a big part in that going forward, and we will continue to evolve, get the design patterns down better, and expand our partnerships to bring that type of information to our users.
One of the big challenges that we are currently focused on helping our clinicians with is information overload and burnout. There is simply an astounding amount of information that comes to clinicians and healthcare practitioners on a daily basis. We are really focusing a lot of our efforts right now on seeing what else we can do to help with this issue. I think a lot of these problems with burnout or information fatigue can be solved with design: we have to keep improving how effective we are at understanding what our user needs and getting them to that specific information quickly.
Our internal studies show that we are pretty good at this now, but there’s always room for improvement. We always get comments from our users saying that we save them precious minutes during the day because of our application, how we write our content, and how we design our features to get people to the information that they need in a quick and efficient way. But as that volume of information keeps growing exponentially, we have to continue to optimize how we deliver the right information to the right user at the right time to avoid that information overload, burnout, and fatigue. This is always at the top of our minds.
Casey has more than a decade of experience in product leadership, where he most recently served as VP of Product at Wiley and led a cross-functional global team supporting subscription revenue of $600 million. He also previously served as VP of Product at WebMD (with oversight of Medscape Education) and SVP of Platform Product Management at Cengage Learning. Additionally, Casey also worked as VP of Product at Elsevier where he oversaw ClinicalKey and several other digital products, including the eBooks platform and the point-of-care content solution (FirstConsult).