TelEMA is an easy to use, low cost telephone assessment platform for clinicians and researchers in psychological sciences and medicine. TelEMA conducts phone surveys that occur during a natural cross section of a participant or patient's everyday life; this type of survey is called an
TelEMA was developed in a collaborative effort between psychologists at Washington University in St. Louis and computer engineers at the University of Illinois in Urbana-Champaign. We have conducted a pilot study to test TelEMA’s effectiveness on a sample of undergraduate students and found TelEMA to be a practical and robust system that enables fast and inexpensive deployment of EMAs. We are currently testing TelEMA on a community sample, and are soliciting feedback from researchers regarding desired features of the TelEMA survey engine and data analysis tools.
We gave a talk about TelEMA and our pilot study at the Association for Behavioral and Cognitive Therapies conference in November 2011. Here are the slides.
You may find our sample materials useful for devising your study and obtaining approval from your Institutional Review Board. Once we have set up a pilot study, you can create a user account on the TelEMA experimenter site.
TelEMA is a telephone-based assessment platform that uses an external web service to send and receive calls and text messages, eliminating the need for special hardware and reducing costs substantially. Researchers and clinicians interact with TelEMA via a web interface, and phone- or text message-based surveys are automatically administered according to an experimenter-specified scheduler and participant preferences. Both the survey engine and the web interface run on a single server and need no special hardware; a single TelEMA server can handle thousands of experimenters and studies at the same time.
By using an external web service to interface with the telephone network, TelEMA eliminates up-front costs for the experimenter and allows studies to include participants from many area codes and countries, using local telephone numbers to minimize cost and inconvenience to the participant. Under TelEMA's pay-as-you-go model, phone numbers can be rented for 1 to 3 dollars per month, and survey calls cost 1 to 3 cents per minute. In the future, we hope to leverage additional features of the telephony service provider, including voice recognition and automatic transcription, to further enhance the experimenter and participant experience.
An experimenter can use our web-based survey builder to design studies consisting of questions with numeric or voice responses. Numeric questions can be open-ended (e.g., asking the participant to count the occurrences of an event) or Likert-type (e.g., scale of 1 to 7). Voice response questions enable the experimenter to gather more qualitative data or assess affect through the character of the participant's voice. Experimenters often want to reorder blocks of questions to prevent participant's from memorizing the order and answering mindlessly, but still want specific instructions to be played before each type of question to guide the participant. To this end, our survey builder allows several types of question order randomization. Survey questions and instructions can be administered using audio recordings uploaded by the experimenter or a text-to-speech engine provided by the web service. While audio recordings will often be easier to understand, text-to-speech can be used to speed up study creation, or to administer participant-specific questions. The short video below contains a few examples of recorded questions.
Experimenters use the same web interface to setup a study using one or more completed surveys. The experimenter can select which surveys are to be administered, in what order, and how often. Additionally, the experimenter can delegate permissions to research assistant user accounts on the web interface to allow only those research assistants to enroll participants and/or view response data. By granting user account permissions on a fine-grained basis, experimenters using TelEMA can ensure that participant data is only exposed on a need-to-know basis and they can comply with Institutional Review Board and other policies.
The experimenter can give participants varying degrees of flexibility in survey timing and reminders, depending on the needs of the study. Participants can enter their normal waking hours into the system, ensuring that they will only receive calls during this time window. Participants can enter a list of phone numbers to be called in succession until one is answered, and can choose to receive reminders via voicemail and/or text message if they miss a survey. The experimenter can designate a time period during which participants can call the TelEMA system back to complete a survey.
Participants enroll in studies using the web interface; currently, the enrollment process is driven by the experimenter or a research assistant, but in future participants will be able to enroll themselves. Participants enter their contact information and preferences as described above, and can complete outgoing and incoming test calls to familiarize themselves with the system. Researchers with the appropriate user account permissions can use the web interface to view compliance data and survey responses as the study progresses. At any time, TelEMA can export a complete dataset, currently as an Excel workbook, organized along several dimensions to be easily exported to other analysis tools. The web-based monitoring tools and exported datasets are useful not only for post-hoc data analysis, but to monitor compliance and survey question effectiveness over time; this data allows experimenters to contact participants with low compliance or modify surveys on-the-fly to maximize the effectiveness of their surveys.
Participant information is secured using technical measures and procedural best practices. We minimize the amount of sensitive information stored in TelEMA and the number of users with access to that information, to minimize disclosure in the event of a security breach. The data that must be stored in TelEMA is encrypted at all times while in transmission and while stored on the server.
For a given participant, TelEMA only stores an ID number, their phone numbers and contact preferences, and survey responses. Notably, their name, demographic information, email address, and any response data from surveys taken outside TelEMA are never input into the system. Furthermore, we tightly restrict access to participant contact information and survey responses; a specially designated administrator account for a given study grants access to these two types of data separately, and for that study only. Access rights can be revoked at any time. Many of these restrictions are required for TelEMA to comply with institutional and legislative guidelines for health-related information, such as those set forth by the Health Insurance Portability and Accountability Act of 1996 (HIPAA).
The TelEMA server is secured with same technical measures used for banking, medical records, and other secure websites. All data transmitted between the TelEMA server and the telephony web service or experimenter computers is encrypted. The TelEMA server's database of phone numbers and survey responses is encrypted. The encryption key is never stored on disk, and must be entered by an administrator when starting the TelEMA application; this procedure ensures that the database cannot be decrypted even if the entire hard drive of the server is compromised. User account passwords for the TelEMA web interface and participant PINs are never stored in plaintext; they are stored and checked using bcrypt. The TelEMA website itself is designed using best practices that mitigate the risk of common security vulnerabilities. The TelEMA server is also protected using a firewall and an intrusion detection system to monitor network traffic. The TelEMA server can only be administered via an encrypted connection authenticated with public key cryptography and a secure passphrase. Finally, the TelEMA server is hosted at a large data center with physical and electronic security measures in place to prevent theft and data loss. These measures in concert reduce the likelihood of a security breach and mitigate the effect if one should occur.