Validation of models for predicting milk urea nitrogen concentrations

The objective of this study was to validate three different models for predicting milk urea nitrogen. 4.749 observations from 855 cows were used. Milk production, body weight, days in milk and lactation number were recorded at milk sampling. Milk was sampled monthly for milk composition analysis. To evaluate the models, they were tested for accuracy, precision and robustness. Model 2 (Kauffman and St-Pierre, 2001) despite being more accurate, was less precise than model 3 (Kauffman and St-Pierre, 2001), while model 1 (Jonker et al., 1998) was the least accurate and the least precise. Models were not robust.


Introduction
Milk urea nitrogen concentration has been used as a management tool to evaluate if a herd (or even the cows individually) has been fed with optimum quantities of protein, if the relation between rumen degraded and undegraded protein is proper and if the balance between protein and energy intake is adequate (Carlsson & Pehrson, 1994).
For this reason, mathematical models have been developed attempting to predict milk urea nitrogen target concentration and thus facilitate its use in the evaluation of nutritional values of diets and in the prediction of nitrogen excreted to the environment (Jonker et al., 1998).These models consider almost all the factors known by affecting milk urea nitrogen concentrations, including nitrogen intake, milk yield and milk protein content.Deviations from this target can identify overfeeding or underfeeding of protein or other issues related to feeding and management.Currently, there are three models which were derived from a model proposed by Jonker et al. (1998) to predict urinary nitrogen.Model 1 for predicting milk urea nitrogen was developed by Jonker et al. (1998), while the other two are more recent (Kauffman & St-Pierre, 2001).In addition, one of the models was adjusted for body weight effect.
After the first model had been developed, Kohn et al. (2002) reported that, in September of 1998, a hardware defect was found on the milk urea nitrogen analyzer that was being used by the DHIA laboratories (Dairy Herd Improvement Association) across the USA to run milk urea nitrogen samples for standard curves.When this defect was corrected, milk urea nitrogen standards changed so that DHIA laboratories reported lower milk urea nitrogen values.
Another important fact is that those models were developed based on experimental databases, from trials carried out in the USA, using their feeds and environmental conditions.So, in order to use them properly, they should be validated under field conditions from different situations and countries.In the case of this study, by using data from a commercial herd, the attempt was to simulate real field conditions.This way, they may provide accurate and reliable results for predicting milk urea nitrogen, which was the objective of the present study.

Material and Methods
In the study, 4,749 observations from 855 Holstein cows of a commercial herd (Brazil) were used.Cows were confined in free-stall barn, with fans and sprinklers, turned on automatically when room temperature reached 23 ºC.After birth, cows were milked in a 2 × 12 herringbone parlor, with AFIMILK ® -SAE AFIKIM data recording system, where each animal was identified and had its milk yield recorded daily.
Cows were fed 7 times/day; the first meal offered at 5 h and the last at 21 h.They received a total mixed ration (48% roughage on a dry matter basis), composed of corn silage, grass haylage, soybean meal, corn germ, high moisture corn grain silage, corn gluten feed, citrus pulp and mineral mixture.Diet was balanced using NRC (1989).The average dry matter intake of the herd was calculated daily by the difference between total feed offered and orts (Table 1).
Data of individual cows -milk yield (kg/day), days in milk (DIM), body weight (BW) and parity and of the herd (dry matter intake) -were recorded on the milk sampling days, which occurred once a month.Milk was sampled, always in the morning milking, and transferred directly to plastic vials (60 mL), each one containing two broadspectrum microtabs of bronopol as preservative, and homogenized for, at least, 15 seconds.They were sent to laboratory for analyses of fat, protein, lactose and total solids, milk urea nitrogen and somatic cells count (SCC).Concentrations (%) of fat, protein, lactose and total solids were determined by infrared absorption (Bentley Instruments, 1995a), SCC (x 10 3 cells/mL) was done by flow citometry (Bentley Instruments, 1995b) and milk urea nitrogen (mg/dL) by an enzymatic and colorimetric methodology (Bentley Instruments, 1998).
Seasons of the year and calving seasons were divided in summer (Nov.-Apr.) and winter (May -Oct.).Fat/ protein ratio was obtained by the division of milk fat by protein percentage.Somatic cells count was analyzed as natural log transformation by the equation ln (SCC+1), because it has no normal distribution (Godden et al., 2001).
The 4% fat-corrected milk was calculated by the equation: (0.4*kg of milk produced) + (15*kg of fat produced), according to the NRC (1989).Then, dry matter intake was estimated by the equation of the NRC ( 2001), as follows: , where: DMI = dry matter intake (kg/animal/day); 4%FCM = 4% fat-corrected milk (kg/animal/day); BW = body weight (kg); LW = lactation week.The 4% fat-corrected milk was used to calculate DMI (NRC, 2001) and the latter was used indirectly in the calculation of the models for predicting milk urea nitrogen, when calculating the nitrogen intake.
The models, evaluated in this study, were derived from a first model proposed by Jonker et al. (1998) to predict urinary nitrogen (UN), as follows: UN = (NI × 0.83) -MN -97; where: UN = excretion of urinary nitrogen (g/animal/day); NI = nitrogen intake (g/ animal/day) and MN = nitrogen secretion in milk (g/animal/day).
Table 1 -Composition of diets and average dry matter intake of the herd to predict milk urea nitrogen (MUN).This way, the equation of the models for predicting milk urea nitrogen included nitrogen intake, milk yield and milk protein content.Additionally, model 3 was adjusted by body weight effect.Predicted milk urea nitrogen concentrations depended on the model used and were obtained by the following equations: -model 1: MUN = UN/12.54(Jonker et al., 1998); -model 2: MUN = UN/17.6(Kauffman & St-Pierre, 2001); -model 3: MUN = UN/(0.0259x BW) (Kauffman & St-Pierre, 2001).
After the first model had been developed, Kohn et al. (2002) reported that, on September 28, 1998, a hardware defect was found on the milk urea nitrogen analyzer that was being used by the DHIA (Dairy Herd Improvement Association) laboratories across the USA to run milk urea nitrogen samples for standard curves.When this defect was corrected, milk urea nitrogen standards changed so that DHIA laboratories reported lower milk urea nitrogen values.Thus, it has been exceedingly difficult to interpret milk urea nitrogen values based on models that were developed previously.Currently, DHIA laboratories use the average milk urea nitrogen reported by several different analyzers to develop standards for indirect methods.
However, more recent studies have suggested that there is a potential bias in this predictor.Kauffman and St.-Pierre (2001) reported milk urea nitrogen equal to UN/17.64 for Holstein cows (model 2).One of the differences between studies that may explain the different coefficients is that laboratory methods used to determine milk urea nitrogen have changed.Another consideration is that the appropriate coefficient depends on BW and so Kauffman & St.-Pierre (2001) developed the model 3.
The purpose of these models is to identify when observed milk urea nitrogen deviates from an expected value, so indicating a potential management problem.By using the 3 different models, expected milk urea nitrogen was predicted from diet and production parameters and these values were then compared with the observed milk urea nitrogen values.
To evaluate the models, they were tested for accuracy, precision and robustness.Accuracy and precision of models were determined by comparison of predicted with observed values.Robustness was determined by comparison of predicted values minus observed values (biases) with other studied factors (Kohn et al., 1998).
According to Kohn et al. (1998), accuracy is evaluated through the mean bias.Bias is the difference between predicted value by the model and observed value and represents the mean inaccuracy of model predictions.Therefore, the most accurate model is the one that presents mean bias as close to zero as possible.Accuracy is measured by the following equation: Mean bias = Precision is a measurement of dispersion between predicted and observed values, i.e., it is the mean variability of the distance between predicted and observed value.It can be evaluated by the root mean square prediction error (RMSPE) or by the residual error.The RMSPE (Bibby & Toutenburg, 1977) is a measure of how much the predictions are well-adequate to the observed values and was calculated by the following equation: RMSPE = However, according to Rodrigues (2002)1 , every time the mean bias is high (lack of accuracy), the lack of precision will be overestimated, i.e., an overestimate of RMSPE, once the mean distance between predicted and observed value also leads to increase in variability between predicted and observed values.Thus, precision is well-evaluated when RMSPE is corrected for lack of accuracy, this way creating residual error, which is defined by remaining error in the prediction model, excluding the error due to mean bias.Residual error is also referred to as the prediction error, excluding the mean bias, obtained by the following equation: Residual error = Mean bias for milk urea nitrogen was estimated by regressing bias (predicted milk urea nitrogen -observed milk urea nitrogen) versus observed milk urea nitrogen and could be used to identify if the magnitude of bias increases, decreases or does not exist with the magnitude of milk urea nitrogen observed values (Bibby & Toutenburg, 1977).
Robustness is the characterization of the model that is less influenced by other selected factors.In order to fit in this concept, the model must have, in relation to the regression straight lines of bias versus variables, low slope coefficients and coefficient of determination (R 2 ) and also lack of slope, indicated by high statistical probabilities (Rodrigues, 2002) 1 .
In order to compare the models with respect to accuracy, mean bias was submitted to analysis of variance (F test) and the comparison of means was accomplished using the Tukey test (5%).For determination of mean bias significance, i.e., if it differed significantly from 0, the T-test was used for mean = 0, using PROC UNIVARIATE (SAS, 1985).To compare the models for precision, residual errors were submitted to test of homogeneity of variance (the test of Hartley) and compared pairwise, using PROC TTEST (Statistical Analysis System, version 5), according to Ott (1993).
Linear bias for milk urea nitrogen was estimated by regression of bias (predicted milk urea nitrogen -observed milk urea nitrogen) versus observed milk urea nitrogen, using PROC REG of the software SAS (Statistical Analysis System, version 5).Robustness was obtained by regression of bias versus selected factors (parity, days in milk, body weight, milk yield, milk fat, protein, lactose and total solid concentrations, natural logarithm of somatic cells count, calving season and season of the year), by PROC REG of SAS (Statistical Analysis System, version 5).Comparison of slope coefficients was accomplished by methods of comparing two straight lines, with the objective to evaluate the interaction between selected variable and models, using analysis of variance (F test) and pairwise comparison by PROC GLM of SAS (1985).

Results and Discussion
Average milk urea nitrogen concentration was 13.15 mg/dL.Cows presented, on average, 662 kg of body weight, 2.3 lactations, 197 days in milk and 34.2 kg/day of milk yield.
The models differed in precision (residual errors); model 3 was the most precise (4.11 mg/dL) and model 1 was the least precise (5.40 mg/dL).The RMSPE were 8.79, 4.60 and 4.35 mg/dL for models 1, 2 and 3, respectively.Thus, despite being the most accurate, model 2 was intermediate precise with residual error of 4.50 mg/dL (Table 2).On the other hand, model 1 was the least accurate and the least precise.
By regressing bias (predicted milk urea nitrogenobserved milk urea nitrogen) versus observed milk urea nitrogen, negative linear biases (slope coefficients) of -0.9540, -0.9673 and -0.9320 mg/dL were found (P<0.0001) for models 1, 2 and 3, respectively (Table 2).This means that bias was the lowest when observed milk urea nitrogen was the highest.However, the linear bias of model 3 differed from model 2 (P = 0.0390), but not from model 1 (P = 0.3039), the same way linear bias of model 2 did not differ from model 1 (P = 0.5541) (Figure 1).
The 3 models were influenced by almost all studied variables (P<0.01).However, model 3 was not affected by logarithm of somatic cells count (P = 0.4204), or by calving season (P = 0.2194) (Table 3).By analyzing R2 , fat and total solids concentration and fat/protein ratio showed the highest values and may explain variations in the models.In addition, they also showed the highest slope coefficients, yet including lactose concentration and season of the year, despite their low R 2 . 1 Natural logarithm of (SCC+1).
2 Slope coefficient of a straight line y = a + bX, by regressing bias (predicted milk urea nitrogen -observed milk urea nitrogen) versus variables.
3 Comparision between slope coefficients of straight lines, where different letters within row differ by the methodology of straight lines comparison (analysis of variance) (P<0.05). 4Coefficient of determination and statistical probability of the analysis of variance to indicate existence of slope coefficient (F test).
Table 3 -Evaluation of robustness (slope, R 2 and probability) for models 1, 2 and 3, including other variables