An Evaluation of Readability of Information on the Internet regarding Total Ankle Replacement

Received: 04-Mar-2021 Accepted Date: Mar 21, 2021 ; Published: 31-Mar-2021, DOI: 10.37532/1897-2276.2021.16(1).11

This open-access article is distributed under the terms of the Creative Commons Attribution Non-Commercial License (CC BY-NC) (http://creativecommons.org/licenses/by-nc/4.0/), which permits reuse, distribution and reproduction of the article, provided that the original work is properly cited and the reuse is restricted to noncommercial purposes. For commercial reuse, contact reprints@pulsus.com

Abstract

Background: Health literacy is the ability to understand basic health information and use this information to enhance health. To ensure accessibility, it's suggested that health literature has a reading grade level equivalent to the American sixth grade. However, previous studies show that this rarely occurs. Total Ankle Replacement (TAR) technology is a complex procedure that may be challenging to explain to the patient. Thus, many patients will turn to the internet for more information and as a ‘quasi-second opinion’. The study aims to evaluate the readability of information on the internet with regards to Total Ankle Replacement. Material and Methods: 110 websites from the two main search engines (Google and Bing) were assessed using the terms ‘total ankle replacement’ and ‘total ankle arthroplasty. Once exclusion criteria were applied, 36 unique websites were categorized and underwent analysis using readability software (WEB FX readability tool). The websites were assessed for readability using the Fleisch Reading Ease Score (FRES) and the Reading Grade Level (RGL). A score of greater than 65 for FRES and an RGL of six or less were considered acceptable. Results: The mean FRES score was 54.95 (SD: ± 13.2); this was significantly below the recognized acceptable standard score of 65 (P<.0001). An ANOVA conducted showed a significant difference between FRES scores based on categories(P=.041) with post-hoc testing showing that the difference between commercial and non-physician scores was the most significant (P=.016; CI:3.84-61.66). The mean RGL was 8.31 (SD: ± 1.95). One-way t-tests showed that these scores were significantly higher than the acceptable standard (P<.0001; CI:1.64-2.97). ANOVA testing showed a significant difference based on category (P=.028) with post hoc testing showing a significant difference between nonphysician and commercial scores (P=.012, CI:0.71-9.33). Conclusion: The majority of the websites are beyond the comprehension levels of the general public. This may lead to serious ramifications for consent as well as post-operative compliance

Keywords

ankle replacement, internet, orthopaedics, readability, health consumers, patient outcomes, total ankle replacement, health literacy

List of Abbreviations

TAR: Total Ankle Replacement, USDHSS: United States Department of Health and Human Services, RGL: Reading Grade Level, FRES: Flesch Reading Ease Score, NIH: National Institute of Health.

Introduction

Painful ankle osteoarthritis affects approximately 1% of the adult population and is associated with similar levels of psychological distress and physical limitations as having previously been associated with hip and knee arthritis [1-2]. Approximately 70%-80% of symptomatic ankle arthritis is post-traumatic, usually associated with a previous ankle fracture [3-5], though repetitive ligamentous injuries have also been linked to long-term degeneration of the ankle joint [1,5-6]. Other rare secondary causes of ankle osteoarthritis include rheumatoid diseases, hemophilia, hemochromatosis, avascular necrosis, and post-infective states [7-8].

Ankle arthritis is a major disease with a deeply negative impact on a generally young and active population [9]. Recent research shows that high demand younger subjects with end-stage ankle arthritis have worse quality of life scores and the degree of physical impairment associated with ankle osteoarthritis, as measured with the SF-36, is equivalent to that reported to be associated with severely disabling medical problems including end-stage kidney disease and congestive heart failure [9-10].

For most patients, ankle arthritis will present insidiously and nonoperative treatment such as bracing, anti-inflammatory medications, and intra-articular injections may help to provide pain relief and improve function [11-14]. As arthritis progresses and conservative treatments fail, patients will turn to surgical options such as Total Ankle Arthroplasty (TAA) to seek long-term pain relief and functional restoration [5].

The primary indication for Total Ankle Replacement (TAR) is endstage ankle arthritis [15]. Previous literature reviews have shown that TAR is a good treatment option for end-stage arthritis with the ability to restore function and mobility as well as relieve pain [5, 16- 22]. Concerns regarding long-term function and mobility with ankle arthrodesis combined with the growing success and evidence in hip and knee ankle arthroplasty have sparked a revolution in developing an equivalent procedure for the ankle [5,23,24]. Advances in implant design and surgical technique have yield better outcomes in recent years; current literature shows a five-year failure rate of 10% for TAR compared to a 50% failure rate previously associated with firstgeneration TAA implants [5,23,25,26]. Further analysis shows a 77% survival rate at 10 years [25-26]. TAA is also associated with an improved postoperative range of motion [16-22] and reduced rates of primary revision compared to ankle arthrodesis [22, 27-29].

Despite these advances, TAR is not without complication. Residual pain is frequent, affecting between 23%-60% of recipients [26], and this must be communicated to patients pre-operatively within the parameters of their consent. Also, TAA works best in older, lower-demand patients, meaning that those who were active or sporty before their arthritis became debilitating may not benefit from this surgical option [1, 5, 30,31]. TAR also requires a demanding post-operative regime that requires strict compliance; this includes plaster cast immobilization of the limb for six weeks postoperatively [1, 30-32].

As TAR is such a complex operation that is still undergoing development and innovation and is still associated with many biomechanical and surgical complications, it can be difficult to explain this procedure succinctly to the patient. Many patients may become confused or overwhelmed by the information they are being given but may be too embarrassed to seek clarity. Often, these patients will instead access the internet as a means to further understand their procedure and to seek a ‘quasi-second opinion’ [33]. Thus, it is of the utmost importance that the information on the internet is as inclusive and accessible to patients as possible to ensure adequate health literacy.

Health literacy is defined as the ability to understand basic health information with such competence as to be able to use this information to enhance health [34]. Lower levels of health literacy are associated with higher inpatient hospital service utilization and increased postoperative complications and costs [35-41]. A lack of health literacy has been shown to significantly impact post-operative rehabilitation compliance, which is particularly pertinent in the case of TAR [39, 30-32]. While simplistic, the key to improving health literacy is to ensure that the materials provided to the health consumers are easily understandable [42]. When we consider that the majority of Americans read at an 8^th-grade level or lower, it is not difficult to discern that a complex procedure such as TAR may be beyond their comprehension, resulting in negative outcomes [42-45]. A failure to understand this completely will reduce compliance and affect post-operative complication and patient satisfaction rates [39-41]. For all of these reasons, the United States Department of Health and Human Services (USDHSS) and the National Institute of Health (NIH) has previously recommended that health education material be written at a Reading Grade Level (RGL) of no higher than the sixth grade in a bid to increase accessibility [42,43]. However, previous studies have shown that this level is often exceeded [42, 46-50].

We conducted an extensive literature search and have not found any previously published study which has sought to determine the accessibility of information about TAR on the internet. The aim of this study, therefore, is to evaluate the readability of information on the internet with regards to Total Ankle Replacement/Arthroplasty.

Materials and Methods

On a single day in June 2020, we searched for websites on Google and Bing using the terms total ankle replacement and total ankle arthroplasty [51]. For each of these searches, the first three pages of returned hits were evaluated (n=110). The reasoning behind this methodology and limit was based on evidence provided by previous studies which have shown that most people do not look beyond the first two pages of website hits and that the majority of people only look at the first page of hits [51,52]. The figures for the returned hits for each search engine and each search term are shown in Table 1.

Table 1. Results from searches performed.

Duplicate websites were removed first and then the remaining websites were accessed and evaluated based on the exclusion criteria. Medical journals, sites requiring logins or composed solely of videos were excluded. This is per previous studies which felt that medical journals would be beyond the capacity of the majority of the population [53]. Following this review of the initial 110 websites, 36 unique webpages underwent further in-depth analysis. A breakdown of this aspect of the methodology is shown via Flow Diagram in Figure 1.

Figure 1:Flow Diagram of methodology for screening websites: Internet search flow diagram, based on the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analyses) statement.

The next step in the analysis was to categorize the 36 included websites by type; following the methodology used in previous studies these categories included academic, physician, non-physician, commercial, media and news, social media, and non-specified [51-53]. Any website linked to a university or teaching hospital was considered academic while physician websites referred specifically to any private website owned by a doctor in private practice. Non-physician referred to websites created by other multidisciplinary team members such as physical therapists, radiographers, and occupational therapists. Commercial denoted websites which contained advertising or were trying to sell products. Social media was added as a category to acknowledge the increased influence of Facebook, Instagram, Tinder, and Tik Tok in the modern era [51-53]. Finally, sites that did not fall into any of the above categories were classed as unspecified. A list of all included sites is available to review in Appendix 1.

Once classified, the websites were transferred to the online readability software. While there are several types available on the internet, the WEB FX readability software was used in this study. Once uploaded, the readability software was used to calculate and produce a Flesch Reading Ease Score (FRES) and a Reading Grade Level (RGL) for each website [46,54]. The FRES is defined as a determination of how difficult a passage in English is to understand [46-49,51,53]. Among readability scoring systems, the FRES is the only metric where a higher score indicates increased readability. A FRES score of 65 or greater is considered to be acceptable [46,51]. FRES scores between 30-50 are considered difficult to read and scores of less than 30 are considered exceptionally difficult to read and are on par with Harvard Law Review [46,51]. Table 2 shows a complete breakdown of the FRES scoring system [51].

Table 2. Breakdown of the Flesch Reading Ease Score system. A score of 65 or greater is concerned to be easily accessible to all reading levels [6].

The Reading Grade Level (RGL) is defined as the ‘cumulative score for the readability of a passage. As previously stated, it is recommended that educational materials generated for health consumers be written at no more than a sixth-grade level of education [42-43].

Statistics were performed using SPSS version 26 (SPSS, Chicago, IL) [55]. The level of significance was set as 5%. To determine whether sites with translation services predicted higher readability scores, 2 sample t-tests were used when the data were normally distributed and the Mann-Whitney U test was used when it was not. To determine the difference between categories, ANOVA testing was performed and if this achieved significance, Post-Hoc statistics were undertaken [51,53]. RGL was compared to the 6th-grade standard using a one-way t-test. A score of 65 or higher was determined to be acceptable for the FRES test; this acceptable standard was compared to the findings using a one-way t-test [51,53].

Results

A total of 36 unique websites were evaluated using the readability tool. Only six out of the thirty-six websites assessed had a FRES score greater than 65 (16.6%). The mean FRES index was 54.95 (SD: +/- 13.2), which classifies the majority of pieces as ‘fairly difficult to read (Table 2). The highest category score as shown in Figure 2 was the non-physician websites. Nine (25%) of the websites reviewed had FRES scores between 30-50 suggesting it would require college-level education to be able to read and interpret them. A one-way t-test was performed comparing the FRES mean to the standard; this was significantly below the recognized acceptable index (p<0.0001; 95% CI:-14.52 to -5.58). An ANOVA conducted showed a significant difference between FRES scores based on categories (p=0.041) with post-hoc testing showing that the difference between commercial and non-physician scores was the most significant (p=0.016; CI:3.84-61.66).

Figure 2:Mean scores for FRES as per category.

The mean RGL was 8.31 (SD: +/- 1.95). As shown in Figure 3 the worst RGL scores were in the commercial category while the best were the non-physician websites. This is consistent with the findings in Figure 2 for the FRES. One-way t-tests showed that these scores were significantly higher than the acceptable standard (p<0.0001; CI: 1.64-2.97). ANOVA testing showed a significant difference based on category (p=0.028) with post hoc testing showing a significant difference between nonphysician and commercial scores (p=0.012, CI: 0.71-9.33).

Figure 3:Mean scores for RGL as per category.

Discussion

TAR is a biomechanical complex and technically challenging procedure which, despite advances, requires strict post-operative program compliance and carries a high risk of complications [30-32]. Thus, if patients are to be able to fully consent to this undertaking, they must have access to reliable information at a level that is easily comprehendible [56].

As observed in previous studies regarding health consumers and the internet, this study has demonstrated that the majority of health education websites regarding TAR exceed the comprehension level of the intended audience [46-52,56]. By failing to adhere to the suggested levels of readability, these websites which are meant to assist patients in understanding their procedure will instead create confusion and anxiety, potentiating the risk of patients developing cyberchondria [57].

This will affect levels of adherence with post-operative care, leading to missed appointments, increased complications, increased hospital re-admission rates, and reduced satisfaction with overall treatment outcomes [35-40]. A lack of credible information may affect the patient’s ability to give informed consent in regards to a procedure [41]. As mentioned in the introduction to this article, TAR has an extremely specific post-operative course which includes a significant amount of time non-weight bearing; failure to comply with this due to a lack of understanding and adherence could result in potentially disastrous complications [30-32,39].

It is thus disquieting that the TAR sites evaluated in the study exhibited an average RGL of 8.31, well above those recommended previously [42-43, 58-59]. 84% of the materials which underwent analysis scored above the recommended sixth-grade reading level with roughly 25% of the materials evaluated were scored at readability levels advised for university textbooks. This observation has note only been noted during this specific study but appears as a consistent trend across the literature, supporting a developing worrisome trend that patients will not be able to fully perceive the information regarding TAR [35-41,51].

However, a surprising yet positive aspect that must be noted during this analysis is that although the mean RGL for the TAR websites is above the recommended standard, it is better than the quoted RGL scores for hip and knee arthroplasty in the literature. One paper by Shnaekel et al. analyzed nine sets of patient education material for hip and knee arthroplasty and found them to have a cumulative RGL of 10.5 [60]. Similarly, Polishchuk et al published a study in 2012 that estimated Reading Grade Level using a different readability test, the Flesch-Kincaid. Using this test, their study shows an estimated RGL for arthroplasty materials of 11.1 [61]. These figures suggest that while TAR website Reading Grade Levels may not be compliant with the recommended standards, there is an awareness of the need to simplify the information among Foot and Ankle specialists.

This difference could be speculated to be associated with the fact that most orthopedic surgeons are aware that the majority of patients will know someone within their social or family circles who will have had a hip or knee arthroplasty procedure that they will be able to seek counsel from. However, TAR is a rarer procedure and thus, places more onus on the physician to provide quality information to the patient. This sense of responsibility may be translated into a better quality of information being presented on websites, in a bid to communicate more effectively with potential health consumers.

A second concerning trend noted during the analysis is the low FRES scores for the commercial, academic, and physician sections shown in Figure 2. The scores for the academic websites, which are affiliated with major teaching hospitals, and for physician sites, which are private foot and ankle specialists, are 55.85 and 53.66 respectively. These FRES score, as noted in Table 2, means that these websites are at a level that requires nearly university-level education to be read and understood on the first pass; this is not feasible for many patients and may leave them confused and overwhelmed. The commercial websites have a mean FRES score of 37.97, which means they require higher education to be able to understand these websites. While it must be acknowledged that commercial sites will contain more technical terms than are perhaps found in the other categories, their FRES score is approximately on par with the readability of the score given to Harvard Law Review [46-49,54]. This would be beyond the scope of comprehension for the majority of patients.

There is much scope to further improve the readability of the TAR websites and to make them compliant with the suggested RGL standards. The Agency for Healthcare Research and Quality (AHRQ) has previously advised an approach of assuming all patients and caregivers have difficulty understanding health information and should be communicated within a manner that anyone can understand [62]. Suggested application of this method for health-related education materials includes the use of diagrams and simple words in a simple conversational style while avoiding jargon [62-67]. Videos could be used to provide an additional further understanding of the procedure and its rehabilitation course.

Orthopedic surgeons who have a specialized interest in foot and ankle surgery should undertake the responsibility of creating easily comprehensible, high-quality education materials based on these guidelines. TAR requires a unique perspective in that the material should not only provide information regarding the surgery itself but should also provide in-depth data about post-operative care and rehabilitation. Once created, physicians and hospitals should then evaluate these materials using the widely available readability software. Ideally, they should look for readability software that identifies complex words and offers substitutions; the authors suggest this as most experienced physicians are used to academic script and maybe blinded to what constitutes a simple or complex word. An example of this would be substituting the more complex “components” for the simpler, more readily understood words “parts” [51].

The final suggestion the authors submit for improving readability and accessibility is to have websites offer translation options where possible. Society has become extensively multicultural in the last two decades and it is no longer acceptable to assume that English is the first language of every household or health consumer [51].

This study is the first to consider the readability of the information on the internet in regards to TAR. However, it must be acknowledged that there are shortcomings within the study [51]. Materials on the internet are in constant flux yet we limited our search strings to a single day. Thus, we acknowledge that there may be better websites that were missed as a result. Furthermore, the first three pages of each search engine were evaluated for each search term; while this was in keeping with the guidelines followed in previous studies, it may mean that more accessible and readable websites on later pages have been excluded [41,51].

A final limitation is software used. The readability formulae determine the difficulty of the passage is based on the letters per word, the syllables per word, or the number of words per sentence. This means that everyday words such as ‘replacement’ may generate a higher RGL than words with fewer syllables and letters such as ‘physis’ which is a medical term and would be poorly understood by the general public [51].

Conclusion

In conclusion, TAR is a complex procedure that has seen improved outcomes as implant quality and surgical technique improve. However, TAR is also associated with significant complications and a difficult post-operative rehabilitation process. It is therefore important that the information regarding this procedure is high yield and easily understood by health consumers.

This study has shown that the data on the internet regarding TAR is relatively inaccessible to the majority of patients and caregivers, with readability scores well above recommended levels. Given the imperative role of health literacy in patient outcomes and the increasing usage of the internet among orthopedic patients, a substantial amount of work needs to be done to improve the readability of these websites. Until this improves, physicians should err towards their patients away from the internet.

Acknowledgments

None to be made.

Declarations

• No patients were used or contacted in this study. Thus, ethics is not applicable

• Data is available upon reasonable request to the corresponding author

• There are no competing interests

• No financial funding available for paper

• No ethical conflicts of interest related to paper

• All authors agree to and consent to the publishing of the paper

• Not submitted anywhere else for consideration

• AMCC and RF contributed to the idea for the paper. AMCC performed the data search and extraction and statistics. AMCC and RF contributed to the writing of the paper and RF performed the final paper review.

• No additional author information applicable

REFERENCES

1. Barg A., Wimmer M.D., Wiewiorski M., et al.: Total ankle replacement.Dtsch Arztebl Int. 2015;112:177-184.
2. Glazebrook M., Daniels T., Younger A., et al.: Comparison of health-related quality of life between patients with end-stage ankle and hip arthrosis. J Bone Joint Sur. 2008;90:499-505.
3. Saltzman C.L., Salamon M.L., Blanchard G.M., et al.: Epidemiology of ankle arthritis: report of a consecutive series of 639 patients from a tertiary orthopaedic centre.Iowa Orthop J.2005;25:44-46
4. Goost H., Wimmer M.D., Barg A., et al.: Fractures of the ankle joint: investigation and treatment options.Dtsch Arztebl Int.2014;111:377-388.
5. Greaser M., Ellington J.K.: Ankle Arthritis. J Arthritis. 2014;3:1-5.
6. Valderrabano V., Hintermann B., Horisberger M., et al.: Ligamentous posttraumatic ankle osteoarthritis.Am J Sports Med.2006;34:612-620.
7. Barg A., Pagenstert G.I., Hugle T., et al.: Ankle osteoarthritis: etiology, diagnostics, and classification.Foot Ankle Clin.2013;18:411-426.
8. Treppo S., Koepp H., Quan E.C., et al.: Comparison of biomechanical and biochemical properties of cartilage from human knee and ankle pairs. J Orthop Res. 2000;18:739-748.
9. D'Ambrosi R., Di Silvestri C., Manzi L., et al.: Post-traumatic ankle osteoarthritis: quality of life, frequency and associated factors. Muscles, Ligaments Tendons J. 2019;9:363-371.
10. Saltzman C.L., Zimmerman M.B., O’Rourke M., et al.: Impact of comorbidities on the measurement of health in patients with ankle osteoarthritis. J Bone Joint Sur. 2006;88:2366-2372.
11. Martin R.L., Stewart G.W., Conti S.F.: Posttraumatic ankle arthritis: an update on conservative and surgical management. J Orthop Sports Phys Ther. 2017;37:253-259.
12. Huang Y.C., Harbst K., Kotajarvi B., et al.: Effects of ankle-foot orthoses on ankle and foot kinematics in patient with ankle osteoarthritis. Arch Phys Med Rehabil. 2006;87:710-716.
13. Schmid T., Krause F.G.: Conservative treatment of asymmetric ankle osteoarthritis.Foot Ankle Clin.2013;18:437-448.
14. Barg A., Smirnov E., Paul J., et al.: Management der Sprunggelenksarthrose.Orthopadie Rheuma.2013;16:44-50.
15. Cody E.A., Scott D.J., Easley M.E.: Total ankle arthroplasty: a critical analysis review. JBIS reviews. 2018;6:8.
16. Singer S., Klejman S., Pinsker E.1., et al.: Ankle arthroplasty and ankle arthrodesis: gait analysis compared with normal controls. J Bone Joint Surg Am. 2013;95:24.
17. Flavin R., Coleman S.C., Tenenbaum S., et al.: Comparison of gait after total ankle arthroplasty and ankle arthrodesis. Foot Ankle Int. 2013;34:1340-1348.
18. Goldberg A.J., Sharp R.J., Cooke P.:Ankle replacement: current practice of foot and ankle surgeons in the United Kingdom. Foot Ankle Int.2009;30:950-954.
19. Seaworth C.M., Do H.T., Vulcano E., et al.: Epidemiology of total ankle arthroplasty: Trends in New York State. Orthopedics.2016;39:170-176.
20. Saltzman C.L., Mann R.A., Ahrens J.E.: Prospective controlled trial of STAR total ankle replacement versus ankle fusion: Initial results. Foot Ankle Int. 2009;30:579-596.
21. Roukis T.S., Prissel M.P.:Registry data trends of total ankle replacement use. J Foot Ankle Surg.2013;52:728-735.
22. Lachman J.R., Ramos J.A., Adams S.B., et al.: Revision Surgery for Metal Component Failure in Total Ankle Arthroplasty. Foot Ankle Orthop. 2019;4:1-5.
23. Haddad S.L., Coetzee J.C., Estok R., et al.: Intermediate and long-term outcomes of total ankle arthroplasty and ankle arthrodesis. A systematic review of the literature. J Bone Joint Surg Am. 2007;89:1899-1905.
24. Hendrick R.P., Stufkens S.A., de Bruijn E.E., et al.: Medium-to long-term outcome of ankle arthrodesis. Foot Ankle Int. 2011;32:940-947.
25. Strasser N.L., Turner N.S.: Functional outcomes after ankle arthrodesis in elderly patients. Foot Ankle Int. 2012;33:699-703.
26. Pugely A.J., Lu X., Amendola A., et al.: Trends in the use of total ankle replacement and ankle arthrodesis in the United States Medicare population. Foot Ankle Int. 2014;35:207-215.
27. Terrell R.D., Montgomery S.R., Pannell W.C., et al.: Comparison of practice patterns in total ankle replacement and ankle fusion in the United States. Foot Ankle Int. 2013;34:1486-1492.
28. Prissel M.A., Roukis T.S.:Incidence of revision after primary implantation of the Scandinavian Total Ankle Replacement System: A systematic review. Clin Podiatr Med Surg.2013;30:237-250.
29. Henricson A., Nilsson J.Å., Carlsson A.:10-year survival of total ankle arthroplasties: a report on 780 cases from the Swedish Ankle Register. Acta Orthop.2011;82:655-659.
30. Valderrabano V., Pagenstert G., Horisberger M., et al.: Sports and recreation activity of ankle arthritis patients before and after total ankle replacement.Am J Sports Med.2006;34:993-999.
31. Naal F.D., Impellizzeri F.M., Loibl M., et al.: Habitual physical activity and sports participation after total ankle arthroplasty.Am J Sports Med.2009;37:95-102.
32. Barg A., Henninger H.B., Hintermann B.: Risk factors for symptomatic deep-vein thrombosis in patients after total ankle replacement who received routine chemical thromboprophylaxis.J Bone Joint Surg Br.2011;93:921-927.
33. Mattews J.R., Harrison C.M., Hughes T.M., et al.: Webpage content and quality assessed for shoulder replacement. AM J Ortho. 2016;45 Suppl 1:20-6.
34. Ratzan S., Parker R.: Health literacy. In: National library of medicine current bibliographies in medicine. Bethesda: National Institutes of Health, US Department of Health and Human Services. 2000.
35. Mitchell S.E., Sadikova E., Jack B.W.: Paasche-Orlow MK. Health literacy and 30-day postdischarge hospital utilization. J Health Commun. 2012;17:325-38.
36. Baker D.W., Gazmararian J.A., Williams M.V., et al.: Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Pub Health. 2002;92:1278-83.
37. Scarpato K.R., Kappa S.F., Goggins K.M., et al.: The impact of health literacy on surgical outcomes following radical cystectomy. J Health Commun. 2016;21 Suppl 2:99-104.
38. Sudore R.L., Yaffe K., Satterfield S.,et al.: Limited literacy and mortality in the elderly. J Gen Intern Med. 2006;21:806-812.
39. Baker D.W., Wolf M.S., Feinglass J., et al.: Health literacy and mortality among elderly persons. Arch Intern Med. 2007; 167:1503-1509.
40. Halverson J.L., Martinez-Donate A.P., Palta M., et al.: Health literacy and health-related quality of life among a population-based sample of cancer patients. J Health Commun. 2015;20:1320-1329.
41. Al Sayah F., Qiu W., Johnson J.A.: Health literacy and health-related quality of life in adults with type 2 diabetes: A longitudinal study. Qual Life Res. 2016;25:1487-1494.
42. Sare A., Patel A., Kothari P., et al.: Readability Assessment of Internet-based Patient Education Materials Related to Treatment Options for Benign Prostatic Hyperplasia.Acad Radiol. 2020;27:1549-1554.
43. United States Health Department Services. America's health literacy: why we need accessible health information. An issue brief from the U.S.D.H.H.S. 2008.
44. De Oliveira G.S. Jr., McCarthy R.J., Wolf M.S., et al.: The impact of health literacy in the care of surgical patients: A qualitative systematic review. BMC Surg. 2015;15: 86.
45. Eichler K., Wieser S., Brügger U.: The costs of limited health literacy: a systematic review. Int J Public Health. 2009;54:313-24.
46. Jayaratne Y.S.N., Andreson N.K., Zwahlen R.A.: Readability of websites containing information on dental implants. Clin Oral Impl Res. 2013;25:1319-1324.
47. Vargas C.R., Ricci J.A., Chuang D.J., et al.: Online patient resources for liposuction: a comparative analysis of readability. Ann Plast Surg. 2016;76: 349-54.
48. Wong K., Levi J.R.: Readability trends of online information by the American Academy of Otolaryngology-Head and Neck Surgery Foundation. Otolaryngol Head Neck Surg. 2017;156:96-102.
49. Schmitt P.J., Prestigiacomo C.J.: Readability of neurosurgery-related patient education materials provided by the American Association of Neurological Surgeons and the National Library of Medicine and National Institutes of Health. World Neurosurg. 2013;80 Suppl 5:33-39.
50. O Neill S.C., Nagle M., Baker J.F., et al.: An assessment of the readability of quality of elective orthopedic information on the internet. Acta Orthop Belg. 2014; 80:153-160.
51. McCarthy A., Taylor C.: SUFE and the internet: Are healthcare information websites accessible to parents? BMJ Paediatrics Open. 2020;4:e000782.
52. Kaicker J., Dang W., Mondal T.: Assessing the quality and reliability of Health Information on ERCP Using the DISCERN instrument. Health Care Curr Rev.. 2013;1:1-4.
53. Bruce-Brand R.A., Baker JF, Byrne DP, Hogan NA, McCarthy T. Assessment of the Quality and Content of Information on Anterior Cruciate Ligament Reconstruction on the Internet. Arthroscopy. 2013;29:1095-100.
54. Readability test tool. WebFX.
55. SPSS:IBM Corp. Released 2018. IBM SPSS Statistics for Windows, Version 26.0. Chicago, IL: IBM Corp.
56. Badarudeen S., Sabharwal S.: Assessing Readability of Patient Education Materials: Current Role in Orthopaedics. Clin Orthop Relat Res. 2010; 468:2572-80.
57. Starcevic V., Berle D.: Cyberchondria: Towards a Better Understanding of Excessive Health-Related Internet use. Expert Rev Neurother. 2013;13:205-213.
58. Weiss B.D.: Health literacy: A Manual for Clinicians. American Medical Association. 2003.
59. Health NIo. How to write easy-to-read health materials. National Library of Medicine. 2013.
60. Shnaekel A.W., Hadden K.B., Moore T.D., et al.: Readability of Patient Educational Materials for Total Hip and Knee Arthroplasty. J Surg Ortho Advances. 2018;27:72-76.
61. Polishchuk DL, Hashem J, Sabharwal S. Readability of Online Patient Education Material on Adult Reconstruction Web Sites. J Arthro. 2012;27:716-719.
62. Brega A.G., Barnard J.M.A., Mabachi N.M., et al.: AHRQ Health Literacy Universal Precautions Toolkit, Second Edition. In. United States of America: Agency for Healthcare Research and Quality. 2015.
63. Plainlanguage.gov. Federal plain language guidelines. In. United States of America: Plain Language Action and Information Network. 2011.
64. Simply Put: A guide for creating easy-to-understand materials. Centre for Disease Control and Prevention, Atlanta. 2009.
65. TOOLKIT for Making Written Material Clear and Effective. Centers for Medicare and Medicaid Services (CMS), Baltimore. 2020.
66. Information Management Plain Language. U.S. Office of Personnel Management, Washington, DC. 2012.
67. Clear Communication: Clear & Simple. National Institutes of Health, Maryland. 2018