At The ECG Quiz, we believe that people learning to diagnose ECGs should practice finding every abnormality in a given record, just as with real records. Experienced ECG readers develop a sequence that is used for every ECG:
In the old days, experienced ECG readers would gather for regular conferences to discuss interesting ECG tracings. As you can imagine, they would not always agree with each other. The reason is that while some features of ECGs are objective, others are subjective. For example, the presence of QRS complexes is objective. The presence of abnormalities in the shape of the T wave is sometimes subjective, especially the finding of “non-specific T wave abnormality”.
The ECG Quiz uses a sophisticated scoring system to account for this “uncertainty” to provide users with meaningful analysis results. In the scoring system, the diagnoses assigned to a particular ECG are categorized as Must or Should based on the user-selected level of expertise. For example, a Beginning Learner would be expected to diagnose only “Supraventricular Tachycardia” when given any example of that rhythm. An Intermediate Learner would be expected to diagnose the mechanism of the tachyarrhythmia when the record gives clues, such as “Atrial Tachycardia”, “Junctional Tachycardia”, or “Sinus Tachycardia”. The Must and Should classification varies by expertise and individual ECG. When a user includes a Should diagnosis in their analysis, they get credit for the diagnosis. However, when a user does not include Should diagnoses when they are present, the user is not penalized.
In addition, each possible diagnosis is classified as Objective, Subjective, or Optional. For example, the “Otherwise Normal ECG” diagnosis is categorized as Optional since this is a diagnosis that experienced readers often make when the ECG recording has only one minor abnormality. However, many experience readers chose not to include this diagnosis even in those cases. Since this diagnosis is Optional, users that include it when appropriate will get credit while users that do not will not be penalized. The Objective, Subjective, and Optional classification assigned to a diagnosis is the same for all ECGs in the system. This concept only applies when those diagnoses appear in the “Must” list.
In addition, The ECG Quiz implements a hierarchical diagnosis list where some diagnoses, such as “Atrial Flutter” are further classified as “Atrial Flutter with 2:1 Ventricular Response” or “Atrial Flutter with Variable Ventricular Response”. More general diagnoses, such as “Atrial Flutter” are higher in the diagnosis hierarchy than more specific diagnoses, such as “Atrial Flutter with Variable Ventricular Response”. Keep reading below to see how this mechanism allows users to use more specific diagnoses as their skills improve and will receive credit for doing so.
When comparing your reading with our reading, each diagnosis entered by you and in our reading is graded as one of the following:
For “Must” Diagnoses in our Reading
For “Should” Diagnoses in our Reading
For Diagnoses in your reading that are not in our reading ⇒ False Positive (FP)
As mentioned above, the diagnosis list in The ECG Quiz is hierarchical with more general diagnoses higher in the hierarchy and more specific diagnoses lower in the hierarchy. Using the hierarchy, the scoring algorithm can modify the grading of diagnoses based on three scenarios.
1. If you do not include an Objective “Must” diagnosis that is in our reading, but you include a “Should” diagnosis that is in our reading that is in the hierarchy of the missed “Must” diagnosis:
As an example, consider an ECG that exhibits Atrial Flutter with 2:1 Ventricular Response. At the “Beginning Learning” expertise level, our reading will include “Atrial Flutter” as a “Must” diagnosis and “Atrial Flutter with 2:1 Ventricular Response” as a “Should” diagnosis. If you entered “Atrial Flutter with 2:1 Ventricular Response” in your reading, it would be graded as a True Positive. But since “Atrial Flutter with 2:1 Ventricular Response” is in the hierarchy of “Atrial Flutter”, the “Atrial Flutter” “Must” diagnosis would be graded as a Not Counted even though you did not include it in your reading because you entered the more specific diagnosis!
2. If you do not include an Objective “Must” diagnosis that is in our reading and you include a diagnosis that is on the hierarchy of the “Must” diagnosis that is not in the “Should” list:
As an example, consider the same ECG that exhibits Atrial Flutter with Variable Ventricular Response. At the “Beginning Learning” expertise level, our reading will include “Atrial Flutter” as a “Must” diagnosis and “Atrial Flutter with Variable Ventricular Response” as a “Should” diagnosis. If you entered “Atrial Flutter with 2:1 Ventricular Response”, your diagnosis would be graded as a False Positive because “Atrial Flutter with 2:1 Ventricular Response” is not in either the “Must” or “Should” lists. However since “Atrial Flutter with 2:1 Ventricular Response” is in the hierarchy of “Atrial Flutter”, the “Atrial Flutter” “Must” diagnosis would be graded as a True Positive. Hence, you are given credit (graded with a True Positive) for correctly identifying the Atrial Flutter, but are penalized (graded with a False Positive) for not getting the correct more specific diagnosis.
3. If you include a “Must” diagnosis that is in our reading and you include a “Should” diagnosis that is also in our reading that is on the hierarchy of the “Must” diagnosis:
This is because in our hierarchy, the “Must” diagnosis is a more general term for a particular diagnosis.
As an example at the Beginner Expertise level, consider a narrow complex tachyarrhythmia that will have “Supraventricular Tachycardia” included in our reading as a “Must” diagnosis, but may also have “Atrial Tachycardia” or “Junctional Tachycardia” as a “Should” diagnosis if evidence in the ECG warrants. “Atrial Tachycardia” and “Junctional Tachycardia” are more specific variants of “Supraventricular Tachycardia”. By scoring things this way, you get credit for the more specific diagnosis, but you do not get “double credit” for including both!
You are given credit (a True Positive) for Subjective diagnoses if you make them correctly (that is, the same as our official reading). You are not penalized (not given a false negative) for omitting Subjective or Optional diagnoses even if they are in the official reading. On the other hand, you will be penalized (given a false positive) if you include a Subjective or Optional diagnosis that is not on the official list of diagnoses for an ECG.
You are given credit (a True Positive) for Should diagnoses if you make them correctly (that is, the same as our official reading). You are not penalized (not given a False Negative) for omitting Should diagnoses even if they are in the official reading.
You are given credit (a True Positive) for a Must diagnosis if a subclassification for that diagnosis is included in the submitted diagnosis list.
You may reread an ECG as often as you like. Your statistical summary will show how you have been doing with each reading of an ECG and with each diagnosis you encounter. In addition, you may reset your statistics whenever you like.
Good luck and have fun!