Written text plays a special role in user interfaces. Key information in interaction elements and content are mostly conveyed through text. The global context, where software has to run in multiple geographical and cultural regions, requires software developers to translate their interfaces into many different languages. This translation process is prone to errors - therefore the question of how language quality can be measured is important. This paper presents the development of a questionnaire to measure user interface language quality (LQS). After a first validation of the instrument with 843 participants, a final set of 10 items remained, which was tested again ( N = 690 ). The survey showed a high internal consistency (Cronbach’s α) of .82, acceptable discriminatory power coefficients (.34-.47), as well as a moderate average homogeneity of .36. The LQS also showed moderate correlation to UMUX, an established usability metric (convergent validity), and it successfully distinguished high and low language quality (discriminative validity). The application to three different products (YouTube, Google Analytics, Google AdWords) revealed similar key statistics, providing evidence that this survey is product-independent. Meanwhile, the survey has been translated and applied to more than 60 languages. HighlightsDevelopment of a survey to measure user rated user interface language quality (LQS).The survey shows good psychometric properties. The LQS correlates moderately to usability metrics. A case study is provided to show how the LQS can be used to improve language quality.The LQS has been translated into 60+ languages and applied to several products.