Disorders of Hemostasis |
Departments of Clinical Laboratory (TQ, MG); Hematology-Oncology (OP, EA, CM, SB, BM, PZ, JP, DM) School of Medicine, Pontificia Universidad Católica de Chile
Correspondence: Diego Mezzano, M.D., Department of Hematology-Oncology, School of Medicine, P. Universidad Católica de Chile, P.O. Box 114-D, Santiago, Chile E-mail: dmezzano{at}med.puc.cl
|
|
|---|
Design and Methods: We studied prospectively 280 consecutive patients with MCB and 299 matched controls, using strict inclusion and exclusion criteria. A single physician recorded the clinical data in a bleeding score and estimated the severity of bleeding in clinical categories. Laboratory criteria for the diagnosis of von Willebrands disease (VWD) and platelet function defects were established from reference values derived from controls.
Results: Fifty patients (17.9%) had VWD (type 1VWD=45, type 2=5). Platelet function defects and mild clotting factor deficiencies were found in 65 (23.2%) and 11 (3.9%) patients, respectively. Thirteen (11.5%) patients had combined defects. The remaining 167(59.6%) patients had BUC, with prolonged bleeding time in 18.6% as their only abnormality. All these disorders, including BUC, were clinically undistinguishable. Moreover, no relationship was found between the severity of bleeding and VWF/platelet function variables.
Interpretation and Conclusions: The diagnostic efficacy of a first laboratory testing in patients with hereditary MCB is 40.4%. Most patients have a disease(s) of high prevalence but unknown pathogenesis. Concurrent bleeding disorders in the same patient are frequent. Our results support the proposal that low plasma VWF levels, but also platelet function defects, should be considered risk factors rather than unequivocal causes of hemorrhages.
Key words: bleeding of unknown cause, diagnosis of inherited disorders of primary hemostasis, mucous and skin bleeding, mild bleeding disorders, platelet function study, relative frequency of disorders of primary hemostasis.
Patients with inherited mucocutaneous bleeding (MCB) present several diagnostic problems. Bleeding symptoms are frequent in the population,1–3 but their clinical relevance is difficult to assess. In contrast, some patients with an unequivocal diagnosis of a known disease may be asymptomatic. There are inherent difficulties in diagnosing von Willebrands disease (VWD) and platelet function defects (PFD), the best characterized disorders of primary hemostasis. Most patients with type 1 VWD have no distinctive genetic markers, and the diagnosis rests on decreased plasma von Willebrand factor (VWF) level and function.4 However, genetic and acquired factors cause a wide distribution of plasma VWF levels, which are weakly correlated with bleeding. Moreover, a fraction of non-bleeder individuals have VWF levels below the established normal range, simulating a type 1 VWD.4 As regards PFD, the prevalence of these defects is unknown, although platelet secretion and signal transduction defects are the most frequent.5,6 There is no consensus regarding the standardization and interpretation of in vitro platelet aggregation and secretion studies, the main diagnostic tools.7–9
The type of bleeding in these diseases is frequently undistinguishable from that in patients with mild clotting factor deficiencies.1,10 Moreover, that tissue fibrinolysis makes some contribution to the bleeding is suggested by the effectiveness of antifibrinolytic drugs in many of these patients without evidence of systemic hyperfibrinolysis.11,12 Lastly, many patients with MCB have no identifiable disease13 even after repeated testing. Their disorder has been labeled as an undefined problem7 or normal hemostasis despite the bleeding symptoms.3,14 To date, there have been no systematic studies to determine the characteristics, frequency and pathogenesis of bleeding in these patients.
This prospective study in unequivocal bleeders was intended to determine the relative frequency of mild bleeding disorders, the prevalence and characteristics of patients with bleeding of unknown cause (BUC), and the diagnostic efficacy of an initial, comprehensive laboratory investigation.
|
|
|---|
4 years and
50 years), referred by physicians aware of the objectives of the study, were always interviewed by the same investigator using a standardized questionnaire,3 modified to assess mainly mucous and skin bleeding. The interviewer recorded the current and past bleeding episodes independently of the age at consultation. Briefly, the most frequent and typical symptoms (Table 1) were scored from 0 to 4, according to their frequency, duration, recurrence, and need and type of therapy. Other symptoms, of lesser frequency, less typical of primary hemostasis disorders, or those present only after exposure to risk, were scored 0 (absent) or 1 (present). The recorded data were processed later providing a numerical assessment of bleeding severity (bleeding score, BS). The BS was compared with the insight of the physician who estimated, interpreted and categorized the bleeding severity at the end of the interview into one of five categories of a clinical classification (CC), adapted from Bolton-Maggs et al.15: intense, moderate, intermediate, trivial bleeders and non-bleeders. Intermediate bleeders were those with spontaneous bleeding from diverse sites, but whose hemostatic system had not yet been challenged by major risk factors for bleeding (e.g. surgery, menstruation); those with abnormal bleeding spontaneously or after a single, non-proportionate injury, but without other pathological hemorrhages; and those with repeated bleeding from a single site without a recognizable cause (e.g. epistaxis, menorrhagia). Trivial bleeders usually had a single site of mild hemorrhages. Succinctly, the BS quantified the bleeding according to the patients (or parentss) point of view and the CC reflected the physicians judgment and perception of the severity of the disease. Both assessments also took into account the bleeding history of first- and second-degree relatives, obtained during the interview, or by phone soon after the patient had collected the information. The controls, recruited independently of the patients with similar geographical, ethnic, social, economic and cultural background, were matched for age and sex, and subjected to the same interview and exclusion criteria as the patients, selecting only non-bleeders (n=252) or trivial bleeders (n=47). The population of 299 controls was constituted by: a) subjects undergoing preoperative hemostatic assessment prior to minor, elective, non-inflammatory surgery (hernia, phimosis and beauty spots; n=99); b) healthy school-children, recruited with parental consent (n=161); c) healthy university students and workers at our University (n=39). All hemostatic tests showed no significant differences among these three groups, except for the last one, which comprised older individuals who had slightly higher VWF values, explained by the age-dependence of plasma VWF concentration. Patients or controls who were taking drugs, were pregnant, had infections or concomitant diseases, a platelet count <100,000/µL, hemoglobin <9 g/dL, elevated serum creatinine and transaminases (ALT/AST), and C reactive protein >1 mg dL were excluded. Patients who had used aspirin or non-steroidal anti-inflammatory drugs were studied at least 7 and 3 days, respectively, after consumption of these drugs. |
View this table: [in a new window] [Download PPT slide] |
Table 1. Quantification of symptoms to assess the bleeding severity in patients with mucocutaneous hemorrhages.
|
2.5th percentile) were derived from the 299 controls, disregarding ABO blood type. Type 1 VWD was diagnosed when VWF:Ag and at least one of the functional tests were subnormal. Ristocetin-induced platelet agglutination (RIPA) was measured by aggregometry using washed, normal and patient platelets.17 VWF:FVIIIB was measured as described elsewhere,18 using anti-VWF monoclonal antibody (vW1, provided by RR Montgomery, USA) and peroxidase-conjugated goat anti-human FVIII (Affinity Biologicals Inc., Ancaster, ON, Canada). VWD variants were diagnosed as recommended.19 All patients with VWF:RCo/Ag and/or VWF:CB/Ag
0.7 underwent RIPA tests and multimeric analysis and those with FVIIIC/VWF:Ag
0.7 were tested for VWF:FVIIIB. When indicated, we also applied Sadlers20 proposal for the definitive diagnosis of type 1 VWD, i.e., plasma VWF values
15 IU/dL. The bleeding time (BT) was measured using pediatric and adult devices.13 Upper cut-off values were established in the 299 controls (<7.0 and <11 minutes for children
6 years old and for older individuals, respectively). Platelet aggregation and 14C-serotonin (14C-5-HT) secretion in platelet-rich plasma were evaluated exactly as described previously,13 using ADP (4 and 8 µM), collagen (1 and 2 µg/mL), epinephrine (10 µM), sodium arachidonate (1 mM) and ristocetin (1.2 mg mL). The lower cutoff values for platelet aggregation and secretion were set as the 2.5th percentile in the 299 controls. Platelet aggregation and secretion in response to 10 µM epinephrine and 4 µM ADP showed a biphasic distribution. Fourteen percent of the controls had only a primary or reversible wave of aggregation or a decreased initial velocity of aggregation, associated with absent or insignificant 5-HT secretion. In all these cases the response to 8 µM ADP, collagen and arachidonate was normal. It, therefore, appears that concentrations of 10 µM epinephrine and 4 µM ADP are threshold platelet stimuli to elicit a full response in a significant proportion of healthy individuals. Accordingly, a platelet function defect was diagnosed when the platelet aggregation and/or 14C-5-HT secretion was abnormal in response be two or more agonists, (excepting the above combination of 10 µM epinephrine and 4 µM ADP), or with either two ADP or two collagen concentrations. Defective aggregation or secretion in response to sodium arachidonate, associated with defects with other agonists, was initially attributed to drugs or other unknown substances. These cases were included only if confirmed by a repeated study.
Platelet 5-HT was measured by high performance liquid chromatography-electrochemical detection,21 and intraplatelet ADP and ATP by a firefly luminiscence method22 using a microplate reader (Fluoroskan Ascent FL, Labsystems, Finland). Values below the 2.5th percentile for platelet 5-HT (=278 ng/109 platelets, in 295 controls) and ADP (=1.2 µM/1011 platelets in 248 controls) and above the 97.5th percentile for ATP/ADP ratio (=2.8) were considered abnormal. The clot lysis assay was performed as described by van dem Borne et al.23 Briefly, 100 µL of platelet-poor plasma were mixed with 100 µL HEPES buffer containing thrombin (120 U/L, Sigma Chem, USA), PC/PS/PE vesicles (20 µM), calcium (33.3 mM), and tissue plasminogen activator (100 ng/mL, Actilyse, Boehringer Ingelheim, Germany). Clot lysis was studied by monitoring the change in turbidity during fibrin lysis at 405 nm in a microplate reader (Labsystems iEMS reader, Espoo, Finland) at 37°C. To correct for inter-assay variability, results were expressed as a patient/normal pooled plasma ratio of the time at which 50% of the clot lysis occurred.
Statistical methods
Results are presented as mean±SD or as mean and range. The means were compared by a two-tailed unpaired Students t test or Mann-Whitney test (for normal and non-normal distributions, respectively). Proportions were compared using Fishers exact test or
2 analysis. Pearsons or Spearmans rank correlations were used for data with normal or non-normal distribution, respectively.
|
|
|---|
|
View this table: [in a new window] [Download PPT slide] |
Table 2. Frequency of diagnosis and demographic data of patients and controls.
|
|
View this table: [in a new window] [Download PPT slide] |
Table 3. Site and frequency of bleeding and family history in patients with a positive diagnosis and in those with bleeding of unknown cause (BUC).
|
![]() View larger version (20K): [in a new window] [Download PPT slide] |
Figure 1. Plasma levels of VWF:Ag, VWF:RCo, VWF:CB in controls and in patients with VWD and bleeding of unknown cause (BUC). The horizontal lines correspond to the 2.5th percentile of the 299 controls. These cut-off values were 42, 37 and 39 IU/dL for VWF:Ag, VWF:RCo and VWF:CB, respectively. If the cut-off is set at the 5th percentile, ten additional patients classified as BUC would qualify for a laboratory diagnosis of type1-VWD.
|
-storage pool disease, with low platelet 5-HT and ADP and high ATP/ADP ratio. One patient had the aggregation phenotype of Glanzmanns thrombasthenia, with normal glycoprotein IIb and IIIa in immunoblotting of platelet lysates. Completion of the study in this patient is needed. The patients with PFD had a slightly faster mean clot lysis than controls (patient/normal ratios of 0.97±0.24 vs. 1.06±0.22, respectively, p<0.007). Mean, range and 2.5th percentiles of maximal percentages of aggregation and 5-HT secretion are shown in Table 4. Twenty-one controls (7.0%) had low values of platelet aggregation/secretion without symptoms, and accordingly, must be considered false positives. Eleven patients had mild clotting factor deficiencies: six had mild hemophilia A (FVIII:C levels of 6, 11, 11, 17, 23 and 28 IU/dL), two had mild hemophilia B (FIX:C levels of 19 and 20 IU/dL), two had FXI:C deficiency (27 and 29 IU/dL) and one had a plasma FX:C of 33 IU/dL. One of these patients had a single episode of trauma-induced knee hemarthrosis and another had a retroperitoneal hematoma after abdominal surgery, as unique symptoms of their disorders of secondary hemostasis. |
View this table: [in a new window] [Download PPT slide] |
Table 4. Maximal percentages of platelet aggregation and secretion in patients with platelet functional defects (PFD), patients with bleeding of unknown cause (BUC) and in the control population.
|
All the patients with BUC had plasma clotting factor levels >35 IU/dL. Statistically, these patients had slightly higher concentrations of FII:C, FVII:C and FX:C and slightly lower levels of FV:C, VIII:C, FXI:C and FXII:C compared with the 299 controls (Table 5). The platelet-poor plasma mean clot lysis time was similar in patients and controls (1.03±0.28 vs. 1.06±0.22, p=NS).
|
View this table: [in a new window] [Download PPT slide] |
Table 5. Plasma coagulation factor activity in patients with mucocutaneous bleeding of unknown cause (BUC) and in the control population.
|
![]() View larger version (19K): [in a new window] [Download PPT slide] |
Figure 2. Distribution of the clinical categories (CC), an index of bleeding severity, in all the diagnostic groups and in patients with bleeding of unknown cause (BUC). Type I von Willebrands disease (VWD) (n=39) platelet function defects (PFD) (n=54); VWD+PFD (n=9); clotting factor deficiencies (CFD) (n=7); BUC (n=167); all patients: (n=280). The proportions of patients in each clinical category were not significantly different among the different disorders ( 2). Patients having concomitant PFD+CFD and type 1 VWD+CFD were not included in this Figure, but were considered in the column with all the patients and in the statistical analysis.
|
|
|
|---|
All these hemostatic disorders were clinically similar, independent of their pathogenesis. The physicians assessment of the bleeding severity in CC showed a good correlation with the BS. However, we found no significant correlation between bleeding severity, (CC and BS), and laboratory variables. So, from the standpoint of a clinical practice, the elaboration, application and analysis of a time-consuming and complex BS was not better than an interview with a pre-established questionnaire followed by the classification of the bleeding severity by the physician (CC). It was not easy for the physician to classify some patients into each CC due to the contrasting perceptions of the patient and physician regarding the clinical importance of the symptoms. The major difficulty was to discriminate normal and pathological bleeding, e.g., in our nomenclature, to discern trivial, non-significant bleeding from mild, abnormal bleeding. Our study design admitted only trivial or non-bleeder controls, precluding the clinical comparison with patients. The family history was only of relative value in the patient assessment, mainly because a high proportion of the controls had first or second degree relatives who were bleeders, confirming a previous report.2 In this regard, the family history was significantly more informative if first and second degree relatives were concomitantly affected.
The BT, a global test of primary hemostasis disorders, has a low sensitivity for differentiating among these disorders. Accordingly, the test is unnecessary for diagnosis and a complete laboratory work-up should be ordered in patients with inherited MCB.24 Blood type O is overrepresented among MCB patients, mostly in those with VWD. This predominance probably reflects the lower plasma VWF in type O individuals.25 Accordingly, assuming that the low VWF is the bleeding risk factor, we established the reference ranges without considering the ABO blood type, as suggested.26
Fifty patients had VWD, 45 of them had type 1, two had type 2A and three had type 2B. Considering one patient excluded because of thrombocytopenic type 2B VWD, the prevalence of non-type 1 VWD phenotypes in our population is around 12%, lower than the 20–30% reported elsewhere.27 A racial factor may explain this difference, since the Aborigine admixture in our Caucasian population borders 30%.28 It is also possible that patients with VWD types 2 and 3, with more symptomatic forms of the disease, are diagnosed before the age of 4, the limit for inclusion in our study. The overlapping distribution of VWF in normal and patient populations4,19,27 adds difficulty to the diagnosis and obscures the role of VWF in bleeding symptoms. In fact, considering the established cut-off values as the 2.5th percentile, eight non-bleeder controls had VWF levels below the normal range. Possibly, upon repeated testing, some patients with borderline values could meet the laboratory criteria for VWD. However, in our study only ten additional patients would have entered this category if the VWF cut-off value had been set at the 5th percentile. The bleeding severity of the 11 patients with VWF levels <15 IU/dL did not differ from that in the remaining 39 patients with higher levels of VWF. This suggests that bleeding in patients with type 1 VWD may not be explained solely on the basis of low plasma VWF levels. This observed lack of relationship between clinical severity and plasma VWF levels differs from a recent observation.29 This is probably explained by the fact that the study by Tosetto et al.29 included non-bleeder individuals (BS <0 and 0 in quintiles 1 and 2, respectively), with a median VWF over 80 IU–1/dL, whereas our analysis was performed only in bleeders. In fact, when the same correlation analysis included only the 39 patients with isolated VWD as well as the non-bleeder controls, we then obtained similar results to Tosetto et al.29
There is a lack of standardization regarding the optimal number and agonist concentration of platelet functional tests.7–9,30 Moreover, the influence of pre-analytical variables, types of samples and recording instruments, and the lack of consensus about agonist type and concentration and interpretation of results, tend to discredit study conclusions. To counteract these drawbacks, we standardized the assay considering all these factors. The data in non-bleeder controls showed that low platelet aggregation/secretion with 10 µM epinephrine and/or 4 µM ADP constituted insufficient criteria for diagnosis. In contrast, normal aggregation with epinephrine usually predicted normal responses to other agonists. An overwhelming majority of the patients with PFD had just primary aggregation with epinephrine, reversible aggregation with ADP and most of them also had low, delayed initiation or decreased slope of aggregation with collagen and arachidonate. These abnormalities were usually associated with reduced 14C-5-HT secretion. Probably, most of these patients have a defect in platelet signal transduction5 of unknown nature. Interestingly, 18 patients had only defective platelet aggregation with different agonists but normal
–granule secretion, and in these cases the nature of the defect is also unknown. Three patients with PFD had
-storage pool deficiency,31 and one patient, phenotypically, had Glanzmanns thrombasthenia. The mechanism and clinical relevance of the shortened clot lysis time in platelet-poor plasma of patients with PFD requires further study. It could partially explain the benefit of antifibrinolytic drugs in some of these patients.11 Given the lack of relationship between the indices of platelet dysfunction and bleeding severity, similar to plasma VWF, we propose that these mild platelet function defects should also be considered risk factors rather than univocal causes of bleeding. This is supported by the observation that aspirin does not induce bleeding in most healthy individuals despite their acquired PFD.
Almost 60% of the patients had bleeding of unknown cause. All hemostatic tests were within normal ranges, except the prolonged BT in 18.6% of them. The nature, sites and severity of bleeding in these patients were similar to those in patients with an established diagnosis, including those with the lowest values of VWF. This, along with the abnormal BT in a fraction of them, suggest that BUC is a disorder of primary hemostasis. The lower mean platelet 5-HT in this population probably has no clinical meaning. In fact, the overwhelming majority of these patients had normal platelet 5-HT, making it unlikely that a slightly lower mean platelet 5-HT affects platelet function. Moreover, individuals taking inhibitors of 5-HT reuptake have very low levels of platelet 5-HT,32 with normal platelet function and no abnormal bleeding. Lastly, we discarded a storage pool defect in some patients, as reported:33,34 the platelet secretion and adenine nucleotide content were normal in all of them.
The diagnostic and management problems posed by these patients are not new, but have received little attention and study. In outpatient populations with bleeding disorders,14,35 in patients with epistaxis36 or menorrhagia,37 between 48.5 and 83% had no laboratory abnormalities to explain their hemorrhages. Certainly, many of them qualify as having BUC. The possibility that a substantial fraction of patients with BUC could have a known disease upon repeated studies, although unlikely, is being investigated. As shown, the prevalence of BUC is more than twice that of VWD and PFD, the best known disorders of primary hemostasis. In this regard, we hypothesize that currently, BUC represents a distinctive and perhaps heterogeneous group of disorder(s) of high frequency and unknown pathogenesis. The addition of low VWF or PFD in a patient with BUC would unmask or increase the patients bleeding tendency, as described for thrombosis in patients with two or more risk factors.38 This proposal is supported by the finding of combined disease, (VWD, PFD and clotting factor deficiencies) in 11.5% of the patients with a known diagnosis, a figure far higher than expected by chance. Supporting that hypothesis, aspirin intake triggered bleeding in 25% of patients with BUC vs. 0.0% in the controls. The analysis of the BT is also illustrative: 18.6% of patients with BUC had a prolonged BT; this proportion rose to 39% and 41% in those with PFD and VWD, respectively, and to 55.6% in those with VWD+PFD. It is possible that new, yet undiscovered factors, may interact with the known ones as bleeding triggers.
The indigenous admixture of our population could probably explain some of the differences between the results of our study and those performed in pure Caucasian populations. In fact, in our records spanning a period of 26 years, we have diagnosed 17 patients with Glanzmanns thrombasthenia and two patients with Bernard-Soulier defects, but only one patient with type 3 VWD. Furthermore, the relative frequency of type 2 VWD in our population (about 10%) is significantly lower than that reported for pure Caucasian populations.
Currently, we can only speculate on the nature of BUC. Factors derived from the endothelium which could affect platelet-vessel wall interactions (e.g., NO, PGI2), have been shown to prolong the BT,39–41 but their very short half-lives preclude their detection by current ex vivo platelet testing. Increased tissue fibrinolytic activity or yet unknown platelet defects could also participate in the hemostatic failure. Clearly, further investigation and novel tests are needed to reveal the mechanisms of bleeding in these patients.
TQ, JP: participated in the conception and design of the study, in its funding, in the acquisition and analysis of data, critical reviewing of the text and final approval of it; MG participated in the acquisition, statistical analysis and interpretation of data, critical reviewing of the text and final approval of it; OP participated in the standardization of critical techniques, acquisition, analysis and interpretation of data, critical reviewing of the text and final approval of it; EA participated in the standardization of critical techniques, acquisition, analysis and interpretation of data, critical reviewing of the text and final approval of it; CM participated in the standardization and performance of platelet aggregation/secretion studies and their interpretation and statistical analysis, critical reviewing of the text and final approval of it; SB participated in the standardization and performance of VWF measurements and diagnosis of VWD variants, in the acquisition of data, critical reviewing of the text and final approval of it; BM organized the reception of patients and controls, performed all the clotting assays and bleeding times, had an important administrative role in the study, critically reviewed the manuscript and gave her final approval of it; PZ participated in the clinical care of the patients, in the study design, and in improving the multimeric analysis of VWD. She critically reviewed the manuscript and gave her final approval; DM participated in the conception and design of the study, in the funding of the study and data analysis, and wrote the manuscript. A summary of this research was submitted to Haematologica and was presented in a Symposium during the XXII National Meeting of the Spanish Society on Thrombosis and Haemostasis (October 26–28, 2006, Granada, Spain)
The authors declare no conflicts of interest.
Funding: supported by Grants 8010002 (DM, JP and TQ) and 1060637 (DM) from Fondecyt, Chile.
Received for publication September 25, 2006. Accepted for publication January 26, 2007.
|
|
|---|
-granules, platelet factor 4, ß-thromboglobulin, and platelet-derived growth factor. Blood 1979;54:1296-319.This article has been cited by other articles:
![]() |
A. D. Mumford, B. B. Dawood, M. E. Daly, S. L. Murden, M. D. Williams, M. B. Protty, J. C. Spalton, M. Wheatley, S. J. Mundell, and S. P. Watson A novel thromboxane A2 receptor D304N variant that abrogates ligand binding in a patient with a bleeding diathesis Blood, January 14, 2010; 115(2): 363 - 369. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. Serebruany, S. V. Rao, M. A. Silva, J. L. Donovan, A. O. Kannan, L. Makarov, S. Goto, and D. Atar Correlation of inhibition of platelet aggregation after clopidogrel with post discharge bleeding events: assessment by different bleeding classifications Eur. Heart J., January 2, 2010; 31(2): 227 - 235. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. Rodeghiero, G. Castaman, and A. Tosetto How I treat von Willebrand disease Blood, August 6, 2009; 114(6): 1158 - 1165. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Minano, A. Ordonez, F. Espana, J. R. Gonzalez-Porras, R. Lecumberri, J. Fontcuberta, P. Llamas, F. Marin, A. Estelles, I. Alberca, et al. AB0 blood group and risk of venous or arterial thrombosis in carriers of factor V Leiden or prothrombin G20210A polymorphisms Haematologica, May 1, 2008; 93(5): 729 - 734. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||