Prototype Testing: What Are Your Opinions?

Posted by Yifei Yu on April 22, 2022 · 10 mins read

Hello, everyone! How are you?

I am glad to inform you that the first stage of our PhD project is completed with two comprehensive literature reviews. This means that we find a valuable niche in academia where the scientific value of this research is highly appreciated. Now, we are moving towards the next prototype-testing stage.

After reading this update, you will understand:

  • What do we know from the literature?
  • What is the prototype about?
  • What is the plan for prototype testing?

1. What do we know from the literature?

First literature review

The first literature review discussed the current situation of applying Information & Communication Technologies (ICT) to support Circular Economy (CE) in the construction industry. We analysed seven types of ICT-based decision support tools developed to support complex CE-oriented tasks. The academia agreed that the research and development of ICT are important to the successful construction CE transition. However, little effort has been made to integrate relevant technologies into a comprehensive information system by considering the complex landscape of CE and the unique characteristics of the construction industry.

Therefore, we first summarized how these ICT-based tools support circular construction/demolition activities in different life cycle phases of a building asset. Then, we provided a conceptual reference architecture of the future information system, Circularity Information Platform (CIP), showing the potential interactions among different technologies in one digital environment. You might be familiar with Fig. 1. below if you read the research proposal.

Fig.1. Conceptual reference architecture of Circularity Information Platform.

Although seven types of technologies are often applied to support CE transition, we will only focus on Building Information Modelling (BIM), Geographical Information Systems (GIS), and Simulation and Modelling (SM) in the coming research because of the team’s expertise and time restriction. Click here to read the full-text version of the first literature review.

Second literature review

The second literature review tackled the challenge of CE policy-making in the construction industry. On the way towards a circular built environment, governmental policy support is viewed as an important enabler. We reviewed the academic articles investigating policy aspects of CE implementation. Our main finding is that there is a lack of integrated policy-making frameworks to manage construction/demolition projects based on CE. Whereas industrial actors wait for effective policy support, policymakers still puzzle on how exactly CE policies should be developed and implemented in this industry.

Therefore, we proposed an integrated CE policy-making cycle by classifying the literature according to the classical policy-making model following the steps of (1) policy agenda-setting, (2) policy formulation, (3) policy decision-making, (4) policy implementation, and (5) policy evaluation. Furthermore, we exemplify the bi-directional policy-making mechanism by integrating policy translation with policy evaluation as an iterative process (Fig. 2):

Fig.2. Bi-directional policy-making mechanism for CE.

In particular, we identified a lack of policy-making integration regarding contents and processes. To improve the content-wise policy integration, we pointed out that only providing economic subsidies is not sufficient. A CE policy package needs to be developed with diverse policy instruments including harmonised/flexible quality assessment standards for secondary materials. On the other hand, ICT-based solutions are expected to bridge the structural disconnection between policy implementation and evaluation by offering more up-to-date monitoring data of CE material flows at a regional level. Click here to read the full-text version of the second literature review.

2. What is the prototype about?

Based on the solid theoretical foundation, we are now investigating the design, delivery, and usage of the Circularity Information Platform (CIP) in organisations. In this stage, you will be involved to test the prototype. A prototype is an early sample, model, or release of a product built to test a concept or process. In this case, the prototype is a design blueprint of the CIP developed for the recycled concrete supply chain in the Twente region, the Netherlands.

We understand the diversity of the information systems in the built environment and most digital systems do not “talk” to each other because of the interoperability issue. We also believe that a well-established master plan is important for the long-term upgrade and maintenance of the information system. Therefore, we aim to provide a “solution template” guiding the design of the CIP by standing on the shoulders of scientific scholars and industrial practitioners.

Until now, we developed a reference architecture model of the CIP by aligning the sustainable motivations, life cycle processes, CE practices, ICT applications, and hardware at a theoretical level. However, it is unclear how likely the model would contribute to CE implementation in practice. We would never know its applicability until we test it with you. Therefore, you are the “boss” in this stage as the potential users. We aim to understand: “how do you like the CIP?”

Taking the first round of interviews with you as the foundation, we are preparing a CIP prototype for you to test. This prototype has three major components:

  • Data model: the model showing the data structure
  • Application logic model: the model visualising and specifying the functional logic behind each application process
  • User interface model: the model demonstrating how potential users can interact with the application

The reason why we do this is that we would like to involve you in the design processes instead of directly providing you with an end product. In the early development stage, we need to make decisions carefully in terms of what to focus on before it is too late to change. Although the prototype is just a mock-up of all the expected functionalities, we view it as a perfect discussion ground for you based on hands-on experiences. With this round of prototype testing, we will be more confident to keep innovating the meaningful functions in detail. A conceptual user interface is provided in Fig. 3 to give you an idea about what it could be.

Fig.3. Conceptual design of CIP's user interface.

You can also click here to operate it online though it is far from complete.

3. What is the plan for prototype testing?

The preliminary plan for prototype testing has two phases:

  • Individual testing (one hour)
  • Group testing (two hours)

First, you will receive the prototype with two documents including the instruction manual and the evaluation form individually via email. In this phase, the purpose is to allow you sufficient time and space to evaluate the prototype without any distraction. Some basic questions will be asked such as:

“What do you think about the functions that the prototype provides?”

You will first learn to use the prototype and then fill in the evaluation form by yourselves. After we collected all the feedback from the participants, an internal evaluation will happen among the research team at the university. We will make the improvements in the prototype accordingly.

Based on the individual testing results, we will launch another round of group testing by inviting you to the campus. By gathering all the potential users around one table, we will “pretend” to use the information system for a concrete supply-demand case based on an imagined workflow. The actions and decisions taken by you will be observed. You will perform the planned workflow by communicating with each other in a dynamic and interactive manner. We do this because the prototype may not realize the automatic real-time information exchange yet. Also, we hope to find out the problems hidden in the application logic of the prototype when stakeholders use it together at the same time.

Finally, we will revisit the theoretical model with your hands-on experiences and user feedback and answer the major question:

“Does this prototype offer the functions that you want?”

We believe this question will triggeer great discussion on the functionality and feasibility of the CIP and lead us to the next stage of the project. The expected testing period is going to be around September 2022. You will receive more detailed information by then.

Thank you!