The ability and rapid access to execution data and information in manufacturing workshops have been greatly improved with the wide spread of the Internet of Things and artificial intelligence technologies, enabling real-time unmanned integrated control of facilities and production. However, the widespread issue of data quality in the field raises concerns among users about the robustness of automatic decision-making models before their application. This paper addresses three main challenges relative to field data quality issues during automated real-time decision-making: parameter identification under measurement uncertainty, sensor accuracy selection, and sensor fault-tolerant control. To address these problems, this paper proposes a risk assessment framework in the case of continuous production workshops. The framework aims to determine a method for systematically assessing data quality issues in specific scenarios. It specifies the preparation requirements, as well as assumptions such as the preparation of datasets on typical working conditions, and the risk assessment model. Within the framework, the data quality issues in real-time decision-making are transformed into data deviation problems. By employing the Monte Carlo simulation method to measure the impact of these issues on the decision risk, a direct link between sensor quality and risks is established. This framework defines specific steps to address the three challenges. A case study in the steel industry confirms the effectiveness of the framework. This proposed method offers a new approach to assessing safety and reducing the risk of real-time unmanned automatic decision-making in industrial settings.