- Research article
- Open Access
“Publish or perish”—presentations at annual national orthopaedic meetings and their correlation with subsequent publication
Journal of Orthopaedic Surgery and Researchvolume 10, Article number: 58 (2015)
Presentation of research at annual national orthopaedic conferences not only serves as a forum for the dissemination of knowledge but is also often a requirement of orthopaedic training programmes. The expected outcome is publication in a peer-reviewed journal. However, publication rates vary for a variety of reasons. The objective of this study was to determine publication rates of presentations from our local Singapore Orthopaedic Association (SOA) annual scientific meeting (ASM) and some of the potential associated factors. We also compared our findings to equivalent meetings worldwide to assess value of scientific content of various orthopaedic conferences.
All presentations of six SOA ASMs were entered into a database. Using presentation titles, author names and keywords in PubMed and Google Scholar, we determined how many presentations progressed to publication in a peer-reviewed journal. Various comparisons were made to determine factors that could influence publication rates. A comparison with national orthopaedic meetings of America, United Kingdom, Ireland, Australia, Germany, Turkey and Brazil was also conducted.
Excluding the ASMs with less than 4 years of follow-up, the publication rate was 35.8%. Both podium and international presenters were found to have significantly higher publication rates than poster and local presenters, respectively, while basic science and clinical research were found to have equivalent rates. Publication rates from other countries’ national conferences ranged between 26.6% and 58.1%.
We suggest that the quality of a presentation is related to its subsequent publication in a peer-reviewed journal. Our findings support the general consensus that the annual meeting of the American Academy of Orthopaedic Surgeons (AAOS) is the gold standard for the dissemination of orthopaedic knowledge updates and advancements in our specialty. Each national orthopaedic association could determine the ratio of “presentations at ASM” to “publication within five years of presentation” and use this as a measure of their annual conference’s impact on the addition and advancement to the orthopaedic literature. This tool may in turn assist clinicians in determining which meetings to attend.
Presentation of research at annual national and international orthopaedic conferences not only serves as a forum for the dissemination of knowledge  but is also often a requirement of orthopaedic training programmes. In Singapore, it is a prerequisite of the Residency Advisory Committee (RAC) that all orthopaedic residents must have at least two podium presentations at the Singapore Orthopaedic Association (SOA) annual scientific meeting (ASM) during their 6 years of training. It is expected that these presentations of basic science and clinical research will form the basis of a subsequent publication in a peer-reviewed journal. Indeed, the phrase “publish or perish” is still very much to the fore with residents often expected to have a specified number of publications in a peer-reviewed journal prior to exiting. This is seen as the gold standard for disseminating scientific research as published papers will have gone through more stringent peer-reviewed processes than conference presentations. In fact, it is recommended that papers considered for publication should not cite conference abstracts for this reason . Despite this, however, even major orthopaedic textbooks may reference conference abstracts for more than half their content . This makes it important to ensure that the information shared at scientific meetings is as accurate as possible. Although publication rates vary from one conference to another for a variety of reasons, measuring the proportion of abstracts that go on to full publication may be a good way to measure the scientific value of a given conference. The objective of our study was to determine the publication rate for our national orthopaedic meeting, the SOA ASM, and investigate potential predictive factors for subsequent publication. We also aimed to make a comparison to similar meetings worldwide [1,3-11] as a rough marker of the quality of research on display.
Materials and methods
All podium and poster presentations from the SOA ASM of six annual conferences, between 2007 and 2013 with the exception of 2008, were obtained. Abstract titles and author names were then entered into a database. For each abstract, the year of presentation was noted along with whether the work was presented as a podium or poster presentation and whether the work presented was from a local or international source. Orthopaedic subspecialties which were relevant to the abstract were also recorded. Author names were then typed into PubMed in order to identify potential corresponding publications. Abstract keywords were combined with author names, using the Boolean term “AND”, in order to narrow our search down if an author had more than one publication. The process was repeated for all possible keywords until either a match was found or until all combinations were exhausted. The process was then repeated using Google Scholar if a publication was not found. From the available data, Χ 2 testing was then used to explore whether publication rates were influenced by the mode of abstract presentation (podium vs. poster), the source of the research (local vs. international) and the type of research conducted (basic science vs. clinical research). Subspecialties were also explored as potential factors influencing likeliness of publication. Factors that were seen as potential predictors of publication were then entered together into a multivariate, binary logistic regression model to further confirm their predictive value with corrections made for potential confounding effects.
For each published abstract, the month and year of publication were recorded. From this, the time from conference presentation to publication was calculated in months. For articles that were published prior to their respective conference presentations, the time from publication to presentation was calculated in a similar way with the number of months given a negative value and included in further analysis. Some aspects of the data analysis would also discard abstracts that were published before a subsequent conference presentation. The journal in which each full paper was published was also recorded as well as its impact factor (IF) for the year in which the respective article was published. Data on the IF was first sourced from Journal Citation Reports® (JCR)  and then from the SCImago Journal and Country Rank (SJR) website  if the relevant data could not be obtained from the former. If no IF value could be found for a particular journal, it was given a value of zero and included in subsequent analysis. Finally, using binary logistic regression, a comparison of publication rates was made with other countries’ national orthopaedic scientific meetings around the world [1,3-11] in which the relevant data was published in the current literature to compare the value of attending various orthopaedic meetings. For those meetings that had more than one publication relevant to this study, the most recent publication was used.
All statistical analysis was performed using IBM® SPSS® Statistic Version 21.0 (IBM Corp. 2012) which included descriptive as well as inferential statistics. Normal distribution of data was inspected visually with the use of histograms and statistically from z scores calculated by dividing the measured skewness value of a data set by its standard error, z scores between −1.96 and 1.96 indicating normally distributed data . Comparison of two or more normally distributed data sets was used using Student’s t-test and one-way ANOVA, respectively. Mann–Whitney U testing was used to compare data sets that were non-normally distributed. Analysis of dichotomous data was done using Χ 2 tests with odds ratios (OR) calculated along with 95% confidence intervals (CI). Fisher’s exact test was used as an alternative when expected counts were seen to be less than five. Differences were seen as significant when p < 0.05.
A total of 443 abstracts were presented in all the conferences studied, of which 125 (28.2%) were published, 20 (16.0%) prior to presentation with 103 (98.1%) of the remaining 105 within 4 years of presentation (average 16.5 months) as can be seen in Figure 1. The majority of published studies were either retrospective (44.8%) or prospective (29.6%) as can be seen in Table 1. Figure 2 illustrates the publication rates for each annual conference included in our study. This shows a drop-off for the 2011, 2012 and 2013 conferences which, when combined, were shown to have a significantly lower publication rate than the rest of the conferences combined (35.8% vs. 22.4%; OR 1.928; CI 1.269–2.929; p = 0.02).
Tables 2 and 3 summarize the analysis of potential predictive factors for publication. Initial analysis revealed only three potentially significant factors that may predict likelihood for publication. When these factors were put together into a multivariate regression model, they were still found to be significant. From this analysis, it was shown that podium presentations were more than twice as likely to be published compared to poster presentations (OR 2.288; CI 1.118–4.684; p = 0.02) and international presenters were more likely to have their work published than local presenters (OR 1.741; CI 1.067–2.842; p = 0.027). Analysis of the third significant predictive factor suggested that trauma papers were less likely to be published compared to other subspecialties (OR 0.530; CI 0.310–0.904; p = 0.02).
Distribution of time to publication revealed the data to be skewed for the whole group, although they were normally distributed for individual conferences. Overall median time, in months, to publication for all published papers was 13 (range −73 to 76). Subsequent analysis excluded papers that were published prior to conference presentation which increased the median to 16.5 (range 0–76). One-way ANOVA analysis revealed significant differences in the time to publication between different years of the conference (p < 0.001) with rates falling in more recent conferences as shown in Figure 3. Further analysis seemed to show that abstracts that were basic science or clinical research papers, podium or poster presentations or from local or international presenters had no influence over the time to publication (p = 0.137, 0.241 and 0.46, respectively).
Published abstracts were disseminated in 54 peer-reviewed journals overall with the top three being the Journal of Orthopaedic Surgery (Hong Kong) (14.0%), the Singapore Medical Journal (10.4%) and the British Journal of Bone and Joint Surgery (7.2%) as shown in Table 4. Of 125 published abstracts, 112 (89.6%) were found in journals that had IF information which left 13 (10.4%) with no IF information according to the methods used in this study. Of the articles in journals with an IF, 82 (73.2%) had the relevant information found in JCR  with 30 (26.8%) having the relevant data in SJR . Table 5 summarizes these findings. The median impact factor achieved for all published articles was 0.96 (0–25.12) with no differences seen in the annual averages (p = 0.291). Further analysis also showed that abstracts that were basic science or clinical research papers, podium or poster presentations or from local or international presenters had no predictive value in determining the IF of a published paper (p = 0.271, 0.957 and 0.598, respectively).
Comparisons were finally made with the publication rates of other countries’ national orthopaedic association meetings that had relevant published data. These were namely from the American Academy of Orthopaedic Surgeons (AAOS) [4,5], British Orthopaedic Association (BOA) , Irish Orthopaedic Association (IOA) , Australian Orthopaedic Association (AOA) , German Society of Orthopaedics and Trauma Surgery (GSOTS) , Turkish Orthopaedics and Traumatology Congress (TOTC)  and the Congresso Brasileiro de Ortopedia (CBOT) . Table 6 summarizes the publications for all these respective meetings. Compared to other conferences included in this study, publication rates for the SOA meeting showed no significant differences to all except the AAOS  (OR 3.523; CI 2.737–4.534; p < 0.001), BOA  (OR 1.451; CI 1.004–2.096; p = 0.048) and GSOTS  (OR 1.367; CI 1.020–1.832; p = 0.037). During the analysis of this data, the high publication rate of the AAOS  at 58.1% was looked at in more detail. This analysis showed a significantly higher publication rate from the AAOS  than all the rest of the meetings included in this study (p values were all <0.001).
This study showed that, with a publication rate of 28.2% (or 35.8% for conferences with at least 4 years of follow-up), the majority of abstracts presented at the annual SOA scientific meeting will not progress to full publication. This is in keeping with most other major national orthopaedic association conferences, although present literature suggests that rates between meetings vary and range between 26.6% and 58.1% [1,3-11]. It should be noted, however, that the previously mentioned 58.1% was at 10 years post-presentation with the figure for the same AAOS meeting being 49% at 5 years . Our local figure of 28.2% is thus likely an underestimate that can be attributed to the publication rate drop-off for the 2011, 2012 and 2013 conferences as this only gives a 3, 2 and 1-year post-presentation period, respectively, to check for publications when in fact we have shown that the majority of publications occur within 4 years and other studies quoting publication rates up to 10 years . This can be clearly understood by noting the 40% publication rate from the 2010 SOA ASM at 4 years. Interestingly, however, a significantly shorter time to publication of presentations from these same three mentioned years compared to earlier years was noted and may be worth further investigating as this may reflect research of higher quality.
While we were unable to calculate statistical significance because of the unavailability of the necessary data, we did note that the majority of published studies were either retrospective or prospective. Podium presentations and international presenters were found to have a higher chance of being published than poster presentations and local presenters, respectively. Our findings regarding podium presentations being twice as likely to be published compared to poster presentations support data from other meetings which show twice  and thrice  the chance of the same. While trauma papers were found to be less likely published compared to other subspecialties, this is especially surprising considering that while general orthopaedic meeting publication rates vary as mentioned earlier, subspecialty conferences have been found to quote publication rates ranging between 40% and 64% [15-18], the highest being in trauma .
The median time to publication after presentation was found to be 16.5 months and ranged between 0 and 76 months. In contrast to other studies , we found that none of the potential factors we analysed, including type of study, source or form of presentation, influenced this. With regard to impact factor, the median achieved for all published articles was 0.96 and ranged between 0 and 25.12. Again, we found that none of the potential factors we analysed had any predictive value in determining the IF of a published paper but could not find anything else in the literature to support our observation.
At first glance, the average publication rate of our SOA ASMs of 28.2% may seem lacklustre. It must be remembered that this figure is likely an underestimate for reasons previously mentioned and evidenced by a 40% figure in 2010. Furthermore, in comparison to seven other worldwide conferences included in this study, publication rates for the SOA ASM showed no significant differences to all except the AAOS  and GSOTS . Again, this may not be a true reflection and may be attributed to the drop-off also previously mentioned. Table 7 shows this to be taken into account by excluding publication rate data from the SOA meetings of 2011, 2012 and 2013. This shows the publication rate from the remaining meetings to be 35.8%, a figure that is comparable to most national meetings within this study [1,6-9], including the BOA  and GSOT  as well as IOA  meetings and significantly higher than the publication rate for CBOT , although it was still found to be significantly lower than the AAOS meeting . When looked at in more detail, further analysis unsurprisingly showed a significantly higher publication rate from the AAOS than all the rest of the meetings included in this study, thus potentially representing that perhaps the annual meeting of the AAOS has sole superior publication rates.
While a number of barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings exists , there is no reason why all national orthopaedic associations should not strive to achieve having the majority of presentations at their annual conferences published in peer-reviewed journals as in the cases of other major orthopaedic  as well as subspecialty meetings [18,20-23]. In our local context, planning of research with a focus on a non-trauma-related prospective or retrospective case series and ensuring one’s submission is accepted as a podium presentation that maximizes the chances of conversion to a publication in a peer-reviewed journal.
Perhaps the publication to the 5-year post-presentation ratio is something each national orthopaedic association around the world should determine as a measure of their annual conference’s impact on the addition and advancement to the field of orthopaedic literature. This tool may in turn assist clinicians in determining which meetings to attend. Our findings support the general consensus amongst many orthopaedic surgeons that the annual meeting of the AAOS is the gold standard for the dissemination of orthopaedic knowledge updates and advancements in our specialty. This statement is easily backed by their superior publication rates of 49% at 5 and 58% at 10 years . It should however be noted that, as many studies presented will not pass the scrutiny of peer-review, the information presented at the AAOS annual meeting (or any other meeting for that matter) should not be used as the sole guide to clinical practice . Finally, with the recent introduction of the Accreditation Council for Graduate Medical Education-International (ACGME-I) and its residency programmes in Singapore, perhaps we should also adopt the stringent guidelines for acceptance of papers to the AAOS annual meeting when reviewing abstracts for the SOA ASM. This should improve the quality of the scientific work being presented at a local level as well as raise the standards of orthopaedic research being performed in Singapore. Lastly, working with the ACGME-I and aiming to have collaborative meetings with the AAOS in Singapore with the hope of learning from our American counterparts and striving to raise our standards of research to match their own is a goal well within reach and well worth pursuing.
We suggest that the quality of a presentation is related to its subsequent publication in a peer-reviewed journal. Our findings support the general consensus that the annual meeting of the AAOS is the gold standard for the dissemination of orthopaedic knowledge updates and advancements in our specialty. Each national orthopaedic association could determine the ratio of “presentations at ASM” to “publication within five years of presentation” and use this as a measure of their annual conference’s impact on the addition and advancement to the orthopaedic literature. This tool may in turn assist clinicians in determining which meetings to attend.
O’Neill BJ, O’heireamhoin S, Byrne AM, Kenny PJ, O’Flanagan SJ, Keogh P. Publication rates of presentations at the Irish Orthopaedic Association annual meeting. Ir J Med Sci. 2014;183:111–6.
Uniform requirements for manuscripts submitted to biomedical journals. Preface. Lancet. 1979;1(8113):428–30.
Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84:615–21.
Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468(5):1428–35.
Voleti PB, Donegan DJ, Kim TW, Lee GC. Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg Am. 2013;95(1), e2.
Ul Haq MI, Gill I. Observational analysis of BOA free-papers (2001): from presentation to publication and comparison with the American Academy of Orthopaedic Surgeons (AAOS). Injury. 2011;42(4):418–20.
Harris IA, Mourad M, Kadir A, Solomon MJ, Young JM. Publication bias in papers presented to the Australian Orthopaedic Association Annual Scientific Meeting. ANZ J Surg. 2006;76(6):427–31.
Schulte TL, Trost M, Osada N, Huck K, Lange T, Gosheger G, et al. Publication rate of abstracts presented at the annual congress of the German Society of Orthopaedics and Trauma Surgery. Arch Orthop Trauma Surg. 2012;132(2):271–80.
Yalcinkaya M, Bagatur E. Fate of abstracts presented at a National Turkish Orthopedics and Traumatology Congress: publication rates and consistency of abstracts compared with their subsequent full-text publications. Acta Orthop Traumatol Turc. 2013;47(4):223–30.
Ejnisman L, Gomes GS, de Oliveira RG, Malavolta EA, Gobbi RG, de Camargo OP. Publication rates of papers presented at the Brazilian Orthopedic Meeting. Acta Ortop Bras. 2013;21(5):285–7.
Hamlet WP, Fletcher A, Meals RA. Publication patterns of papers presented at the annual meeting of the American Academy of Orthopaedic Surgeons. J Bone Joint Surg Am. 1997;79:1138–43.
ISI Web of KnowledgeSM. Journal Citation Reports®. Secondary ISI Web of KnowledgeSM: Journal Citation Reports® 2014. http://admin-apps.webofknowledge.com/JCR/JCR?PointOfEntry=Home&SID=R2T9Yp5yVAdxgXM95VH.
SCImago Journal and Country Rank. Secondary SCImago Journal and Country Rank 2014. http://www.scimagojr.com/index.php.
Doane DP, Seward LE. Measuring skewness: a forgotten statistic? J Stat Educ. 2011;19(2):1–18.
Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major Spine Specialty Society meetings (NASS, SRS, ISSLS). Spine. 1999;24:425–7.
Jackson KR, Daluiski A, Kay RM. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America. J Pediatr Orthop. 2000;20:2–6.
Preston CF, Bhandari M, Fulkerson E, Ginat D, Egol KA, Koval KJ. The consistency between scientific papers presented at the Orthopaedic Trauma Association and their subsequent full-text publication. J Orthop Trauma. 2006;20(2):129–33.
Nguyen V, Tornetta 3rd P, Bkaric M. Publication rates for the scientific sessions of the OTA. Orthopaedic Trauma Association. J Orthop Trauma. 1998;12:457–9.
Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta 3rd P, Cook DJ. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am. 2003;85:158–63.
Daluiski A, Kuhns CA, Jackson KR, Lieberman JR. Publication rate of abstracts presented at the annual meeting of the Orthopaedic Research Society. J Orthop Res. 1998;16:645–9.
Harris IA, Mourad M, Kadir A, Solomon MJ, Young JM. Publication bias in abstracts presented to the annual meeting of the American Academy of Orthopaedic Surgeons. J Orthop Surg (Hong Kong). 2007;15(1):62–6.
Kleweno CP, Bryant WK, Jacir AM, Levine WN, Ahmad CS. Discrepancies and rates of publication in orthopaedic sports medicine abstracts. Am J Sports Med. 2008;36(10):1875–9.
Yoo S, Oh G, Wang JC. Publication rates of presentations made at annual meetings of the American Orthopaedic Society for Sports Medicine and the Arthroscopy Association of North America. Am J Orthop (Belle Mead NJ). 2002;31(6):367–9.
The authors declare that they have no competing interests.
ZJD came up with the study, collected and collated all the data and was the lead author in terms of writing the manuscript. SSH and KLW assisted in keying in the data, analysed it and managed the statistics. PYN was the main compiler of the data and assisted in putting together a spreadsheet of it. DPM participated in the design of the study and helped draft the manuscript. All authors read an approved the final manuscript.