The aim of the study was to determine whether different ultrasound probe/transducer configurations produce the same measurements. Two investigators undertook 10 scans of a general purpose semi-solid multi-tissue ultrasound phantom (phantom A) using two ultrasound scanners with a linear and curvilinear probe. From those 10 scans, two measurements of cross-sectional area (CSA), width and thickness were made. These measurements were then repeated with an open-top fluid-filled phantom, with 10% ethanol solution (phantom B). Intra- and inter-rater reliability were examined using Bland and Altman plots. Agreement between measurements made with the two probe types was also assessed using Bland and Altman plots. An independent samples t-test was used to compare statistical differences between probe type configuration. There was a significant difference (p < 0.05) and a tendency for increased measurements in CSA and width, and decreased measurements in thickness when using a curvilinear probe on phantom A, and these differences were not scanner specific. When imaging phantom B there were no significant differences in measurements between probe configurations; however there was a small bias for smaller CSA measurements with a curvilinear probe. In conclusion there are small differences in measurements obtained from different ultrasound probe configurations using a semi-solid phantom, but their clinical significance is unknown.