Search strategy
The selected websites were chosen based on the results of a national survey that identified web-sources commonly used by Israeli school adolescents when seeking OHI [41]. According to the survey, over 70% of students indicated having sought health information through Google.co.il, Finder, and Kafe. Five other search engines (Tapuz, Nana10, Walla, Bing, and Yahoo) were also mentioned in relation to searching for OHI, but less frequently. Therefore, several searches were conducted using identical search terms in Hebrew using Bing, Finder, Google, Kafe, Nana10, Walla, and Yahoo. Tapuz was excluded as it searches only within the Tapuz website itself.
We gathered the first 100 search results for each search term [various Hebrew permutations of birth control pill, birth control pills, pregnancy prevention pills, and contraceptives] on 6 search engines. The seventh search engine–Nana10, produced fewer than 30 results per search term and therefore all of Nana10’s results were added to the list. To increase the number of possible websites to be assessed and in order to avoid duplicates, the results were organized into four groups based on the type of search engine: 1) Google and Google-powered search engines–Finder and Kafe, 2) Yahoo-powered search engines–Yahoo and Walla, 3) Bing, and 4) Nana10. For each search term, we compared the results from the four search engine groups and excluded overlapping items. A list of 158 websites was compiled, after combining the results of the search of the six search-terms and excluding non-Hebrew websites. Of these, 27 were unique (i.e., not duplicates), accessible, active, and provided (non-user-generated) OC information on at least one webpage. This list contained the websites of two of the four major Israeli HMOs (Health Management Organizations – Kupat Holim), so the websites of the other two HMOs were added in order to assess the quality of all four HMOs. Thus, 29 websites were selected for evaluation. Within each website, text searches were conducted to identify all the pages included in the same domain name with at least one title and one paragraph with information on OC use. When available, non-OC related pages of the website (such as About pages, Contact pages, and pages with information on Privacy policy) were also evaluated to assess parameters of credibility (e.g., website’s funding, authors, and age policies) and usability (e.g., consistency in design and function).
Website quality evaluation
The quality of each website was assessed on three key parameters: accuracy/completeness, credibility, and usability.
Accuracy and completeness
Accuracy/completeness of the OC content was assessed based on 13 established content items selected by a family planning expert from contraception guidelines [49]. Items that were misperceived by Israeli physicians in a study on OC misconceptions [47] (perhaps due to inconclusive scientific evidence) were excluded, as we were unable to definitively classify the information as accurate or inaccurate. The final list included items such as efficacy (e.g., of 1000 women using the pill as prescribed, 3 will become pregnant within a year), effectiveness in the first year of use (e.g., about 8% of typical users initiating OC will become pregnant in the first year of use), safety, reversibility, suggested frequency of use (e.g., pills should be taken daily), STI protection (e.g., pills do not protect against STIs), side effects (e.g., headaches, nausea, vomiting, breast pain/tenderness, weight gain, fatigue, mood changes), dose (e.g., take one pill a day until the pack is finished), back-up contraceptive method (e.g., use a backup contraceptive until taking 7 consecutive pills, if missing a pill, if late starting a new pack, or in case of severe vomiting or diarrhea), or instructions in case of missing a pill (e.g., taking another pill within 12 hours will still prevent pregnancy). The 13 items were assessed on a 4-point scale [inaccurate information (−1), information not presented (0), information partially presented (1), and information complete and accurate (2)], and computed on a composite percentage scale ranging from 0 to 100%, with a score of 26 (the maximum possible for this 13-item scale) representing 100%.
Credibility
The HONcode principles were developed to assess the credibility of OHI [35]. The eight HONcode principles address the web-author’s authoritativeness, the extent to which the information supports, rather than usurps, the doctor-patient relationship, stated privacy policy, attribution of the information to its sources, the extent to which the site backs up claims relating to benefits and performance, accessibility of the web-editor, financial disclosure, and advertising policy. Based on these original eight principles, a set of 14 questions was developed, each on a 3-point scale [parameter absent (1), parameter partially addressed (2), and parameter fully addressed (3)], and computed on a composite percentage scale ranging from 0 to 100%, with a score of 42 (maximum possible for the 14-item scale) representing 100%.
Usability
Basic aspects of the websites’ usability were also assessed by applying an 11-item instrument selected from the Minervation Validation instrument (questions 2.1.2-2.1.5, 2.2.1-2.2.3, and 2.3.4) [50] and/or the University of Michigan “website evaluation checklist” (questions 12, 28, 30, 34, and 35) [51]. The items included aspects such as a “return to home page” option on all pages, consistency of functions and design, and appropriateness of content for the target adolescent audience (e.g., readability as assessed by amount and size of text, use of age-appropriate medical and scientific terminology). Each item was assessed on a 3-point scale [parameter absent (1), parameter partially addressed (2), and parameter fully addressed (3)], and computed on a composite percentage scale ranging from 0 to 100% with a score of 33 (maximum possible for the 11-item scale) representing 100%.
Classification of websites
The websites were classified as belonging to one of the following 5 categories: a) HMOs (i.e., websites of the four leading Kupat Holim), b) health portals (i.e., websites that provide information on a wide scope of health and medical subjects), c) contraception-specific (i.e., websites dedicated primarily to informing the public on the subject of contraception), d) promotional and commercial (i.e., websites whose goal is to promote a certain product, service, or brand), e) life style and women’s websites (i.e., websites that provide information on contraception in the context of women’s wellbeing and child-rearing advice).
Reliability analyses
All websites were evaluated by one of us (LF). Following analyses of the data and in response to a reviewer’s suggestion on a previous version of this report, six randomly selected websites (a total of 38 web pages) were independently assessed by YN and RS, and reassessed by LF. Some inter-rater variation was observed for accuracy/completeness and usability (Cronbach’s alpha coefficients–0.84 and 0.73, respectively; Additional file 1). A greater degree of variation was noted for the 15-item credibility scale, which revealed a Cronbach‘s alpha coefficient of 0.66. This value increased to 0.75 upon deletion of one item (“Is there a clear statement regarding the minimum age for the use or participation on the website?”). The credibility parameter for all 29 websites was therefore reanalyzed using the 14-item scale.
Statistical analysis
Mean scores, standard deviations (SD), ranges, and percentages were calculated for all websites and for category-specific websites. Pearson correlation coefficients of the three parameters assessed were estimated for all, and for category-specific websites.
Statistical analyses were performed using SPSS 17.0.