PURPOSE To investigate the accuracy of output factor measurements using a commercial (Exradin W1, SI) and a prototype, "in-house" developed, plastic scintillation dosimeter (PSD) in small photon fields. METHODS Repetitive detector-specific output factor OFdet measurements were performed in water (parallel to the CAX) using two W1 PSDs (SI), a PTW microLion, a PTW microDiamond and an unshielded diode D1V (SI) to which Monte Carlo calculated corrections factors were applied. Four sets of repetitive measurements were performed with the W1 PSD positioned parallel and perpendicular to the CAX, each set on a different day, and with analytically calculated volume averaging corrections applied. The W1 OFdet measurements were compared to measurements using an "in-house" developed PSD in water (CHUQ) and both were validated against a previously commissioned Monte Carlo beam model in small photon fields. The performance of the spectrum discrimination calibration procedure was evaluated under different fiber orientations and wavelength threshold choices and the impact on the respective OFdet was reported. RESULTS For all detectors in the study an excellent agreement was observed down to a field size of 1 × 1 cm2 . For the smallest field size of 0.5 × 0.5 cm2 , the W1 PSDs presented OFdet readings higher by 3.8 to 5.0% relative to the mean corrected OFdet of the rest of the detectors and by 5.8 to 6.1% relative to the CHUQ PSD. The repetitive W1 OFdet measurements in water (parallel CAX) were higher by 3.9% relative to the OFdet measurements in Solid WaterTM (perpendicular CAX) even after volume averaging corrections were applied, indicating a potential fiber orientation dependency in small fields. Uncertainties in jaw and detector repositioning as well as source variations with time were estimated to be less than 0.9% (1 σ) for the W1 under both orientations. The CHUQ PSD agreed with the MC dose calculations in water, for the smallest field size, within 1.1-1.7% before any corrections and within 0.3-0.8% after volume averaging corrections. The spectrum discrimination method provided reproducible Cherenkov spectra under the different calibration set-ups with noisier spectra extracted if the calibration is performed in water and parallel to the CAX. The impact of fiber orientation and wavelength threshold during calibration on OFdet was in general minimal. CONCLUSIONS Clinically relevant differences were observed between similar scintillator dosimeters in photon fields smaller than 1 × 1 cm2 . Further research on PSDs is needed that can explain the origin of these differences especially related to the Cherenkov spectrum dependencies on the optical fiber technical characteristics.