Author: Ben Linders

Artificial intelligence can assist software delivery and be used to automate software testing and optimize project work. Dimitar Panayotov uses ChatGPT to generate test data, create email templates, and produce explanations based on test results. This saves him time that he can invest to become more productive.

Dimitar Panayotov spoke about how he uses ChatGPT in testing at QA Challenge Accepted 2023.

Artificial intelligence (AI) is a collection of algorithms and neural networks that have been fed enough data to assist IT professionals, Panayotov said. That makes it a very advanced helper tool for assisting in the daily life of a quality assurance engineer, from information dumps on specific subjects, test documentation, and case writing, to workflow guessing patterns, he added.

ChatGPT is great at generating test data; this is its biggest strength, Panayotov said. It can also create test strategies based on requirements and based on best world practices, he added. It can write test cases, but it needs additional support because it does not train enough.

ChatGPT can be used for generating email templates and explanations based on test results, as Panayotov explained:

It can give a better human explanation for the test results that are usually only graphs and numbers for non-technical people. Also, it can create tables based on input.

Because the AI model is still in development, there are some bugs and glitches that may occur while using it, as Panayotov explained:

Overfitting with data (the information imputed by the developers of ChatGPT may be incorrect or derived from a common or faulty source) will give incorrect answers or missing values when asked.

Hallucination is another major problem – basically, the model tries to compile an answer based on the string that the user inputs, but because it has some limitations and rules that it has to follow, the answer may come as a complete fiction.

ChatGPT is a more powerful software tool than for example Google or Reddit, or even the all-knowing Stackoverflow, Panayotov said. Use it as a regular tool, know the product limitations, and don’t misuse it for maleficent purposes, he suggested:

It will save you so much time that can be invested to become more productive, for instance by learning new software languages and tools or expanding your working capabilities.

InfoQ interviewed Dimitar Panayotov about how he uses ChatGPT in his daily work.

InfoQ: How do you use ChatGPT for automated testing?

Dimitar Panayotov: I use ChatGPT for many operations that shorten the creation and execution time of automation projects and CI/CD pipelines. Here is a list of functions I use the model for:

• Preparing test plans and test scenarios- the model is very useful when creating new test scenarios based on web, mobile, or Cucumber test cases.
• Creating scripts in different languages- the general knowledge of the model for each programming language is just mind-blowing.
• Designing test cases- it can create specific design patterns for specific business logic based on universal models.
• Creating email templates- generating test data is one of the biggest strengths of the model itself.

InfoQ: What about using ChatGPT to optimize software project work?

Panayotov: Here are some examples of what the model can do for you to help you improve the speed of execution, optimize resource usage, and structure of your project:

• Format data and write documentation- documentation writing for a project is the most boring thing in the life of an IT specialist. The model can generate it, but it only does it if you give it the structure of your project.
• Implement CI/CD pipelines almost from scratch- with a simple request and with specification, the model can generate scripts for execution by almost all software and products that are in the CI/CD family.
• Create specific classes- if asked, the model can generate specific classes for your project based on its structure.
• Code reviews and error handling- the model can do basic code reviews of your code and even find vulnerabilities, but you need to know that your data can be used in future model answer generations.

We have to be sure about whether we want the model to have access to our project and give private information. ChatGPT is an open-source product; it wouldn’t be wise to input any client or personal sensitive information.

InfoQ: What’s your advice for testing with ChatGPT?

Panayotov: Like I always say: work smart, not hard. All manual and data entering and data generating can finally be skipped, as the ChatGPT model is made for that sole purpose, and more. Create as much dummy data and basic project structures as you need. Just don’t add personal or client data, because the use of it can be compromised in future versions or by data mining.

Recognizing the challenges faced by frontend developers, the Norwegian Labour and Welfare Administration decided to build a dedicated platform to address their needs. It offers services like a CDN for static content delivery, an observability stack for monitoring and debugging, and feature management using Unleash. The platform is treated as a product, prioritizing user needs, collaboration, and learning, to drive adoption and improve the developer experience.

Andreas Nordahl and Hans Kristian Flaatten spoke about building a platform for frontend development at NDC Oslo 2023.

In today’s digital landscape, it’s important to provide dedicated platforms to support developers, Flaatten said. The Norwegian Labour and Welfare Administration (NAV) has embraced smaller stable and autonomous teams. These teams are working within a specific subdomain of NAV and own the entire life-cycle of the product they are building, Flaaten mentioned.

The NAV frontend platform is built on top of the existing NAIS application platform. It provides a set of services tailored to frontend development, as Nordahl explained:

We started by establishing a Content Delivery Network (CDN) for efficiently delivering static assets. It is fully integrated with our CI pipelines making it super simple for teams to get started.

After this, observability became the most critical aspect, where frontend applications did not get much support from the existing tools. Nordahl mentioned that they leveraged their existing Grafana-stack to collect frontend logs, exceptions, performance metrics, and tracing with a few adjustments.

Another thing that was added was feature management. Using Unleash, it allows teams to implement and control feature toggles, facilitating safe testing and easy reversion of changes.

The frontend platform offers extensive support for software developers, Flaatten explained:

By providing services such as the CDN, observability stack, and feature management, we empower developers to build, monitor, and optimize frontend applications effectively. We emphasize self-service, ensuring that development teams can utilize the services without heavy reliance on us.

Clear and intuitive interfaces, along with comprehensive documentation and guides, further enhance the developer experience and drive adoption of the platform, Flaatten mentioned.

Nordahl mentioned that, to drive adoption they actively engage with development teams, seeking feedback, addressing concerns, and showcasing the platform’s capabilities. This proactive approach fosters a culture of collaboration and learning, driving adoption and continuous improvement, he said. They provide comprehensive documentation, guides, and cultivate a strong community to support onboarding efforts and raise awareness of the platform’s benefits.

To accommodate different frontend technologies without imposing technology limitations on application teams, they explored emerging standards like Web Components, WebAssembly, and OpenTelemetry, Flaatten mentioned. This approach enabled them to create services that could be consumed by any frontend application, while still allowing teams to make their technology choices.

InfoQ interviewed Andreas Nordahl and Hans Kristian Flaatten about building a platform for frontend development.

InfoQ: What made you decide to build a platform for frontend development?

Andreas Nordahl: Within our environment of over 100 cross-functional product teams, frontend development presented some unique challenges as each of them are responsible for their own presentation to their users. While the reusable frontend components of our design system are valuable, they only address a fraction of the needs of our frontend developers.

Our existing application platform, NAIS, lacked the necessary support required for frontend applications causing more cognitive load on frontend developers. Recognizing these challenges, we identified the need for dedicated frontend platform services to improve the overall developer experience.

InfoQ: What challenges did you face?

Nordahl: One challenge was hiring frontend developers with the necessary expertise to leverage the platform effectively. We recognized that providing tools alone was insufficient; supporting teams in acquiring the required skills was equally important. Alongside our hiring efforts, we focused on fostering a supportive community, creating comprehensive guides and documentation, and developing tools to empower new frontend developers.

InfoQ: What’s next?

Hans Kristian Flaatten: Frontend observability will continue to be a focus area as performance metrics can help teams improve their application’s performance and reduce complexity. Technologies like Web Components offer exciting possibilities for building micro frontends, enabling encapsulated and reusable code that can function independently.

According to Robin Yeman, there are several challenges in building hardware-reliant cyber-physical systems, such as hardware lead times, organisational structure, common language, system decomposition, cross-team communication, alignment, and culture. A solution to such challenges is to apply agile at the systems level, and to architect both hardware and software into modular components.

Robin Yeman spoke about building cyber-physical systems with agile at QCon New York 2023.

When building hardware-reliant cyber-physical systems, we frequently get the software done rapidly, but it has to sit on the shelf until the hardware is ready, Yeman mentioned. Cyber-physical systems often have to manage long lead times for hardware, delaying feedback. Once the hardware is ready we need to update the software because of technology changes during that time, Yeman said.

There are several challenges when using agile for large scale for initiatives building cyber-physical safety-critical systems. Yeman explained that companies are often organised by function, not by product or outcome, leading to multiple handoffs and delays in delivery:

When we try to organise by outcome, the different functions such as hardware engineering and software engineering do not have a common language, making communication hard.

Another challenge Yeman mentioned was that many engineers struggle to decompose the system into small enough modules that allow them to incrementally implement them within timeboxes while building a cohesive piece of functionality that can be validated.

Many organisations experience cross-team communication challenges based on varying priorities, Yemen said. Instead of looking at the system as a whole, each team is only aware of their small portion of the system and are less concerned with timelines from other teams leading to a lack of alignment.

Yemen mentioned that culture still remains to be the biggest challenge for organisations. Large companies that have been established for many years have experienced success in how they have completed work in the past.

In addition, their existing organisation structure and the architecture of the system have significant impact on dependencies:

If you move all of your teams to cross-functional teams but you have not changed your architecture, you could bring everything to a stop. If you change the architecture without updating the organisation, we may increase the number of bottlenecks.

Adapting to apply agile at the systems level requires a change in organisational structure, educating engineers on modular open system decomposition, applying multiple horizons of planning, ensuring all of your teams are on the same cadence, and ensuring you have test environments early in development, Yemen said. Hardware in the loop, software in the loop, and model-based testing environments are often not invested in until much later in the project lifecycle, which reduces our ability to obtain the benefits from Agile.

Yeman suggested intentionally architecting both hardware and software into modular components that can be modified independently, and only connect to other modules through standardised interfaces.

InfoQ interviewed Robin Yeman about using agile for building cyber-physical systems.

InfoQ: What made you decide to explore agile for building cyber-physical systems?

Robin Yeman: In order to deliver capabilities to our customers at the speed of relevance which is the frequency at which customers can take deliveries, we needed to apply agile practices at the system level. As I applied agile practices to larger systems, I found that the majority of practices were agnostic to the type of work to be completed, for example timeboxing or iterative development can be applied to any type of work, from writing a book to building a satellite. The speed of relevancy varies based on the type of system, for example we may need less frequent updates to a weapon system than we do for a phone application.

InfoQ: How do you adapt and apply agile for planning large systems?

Yeman: Agile practices break down into over a hundred practices; we would never use them all, but identifying the specific goals for your system will allow you to pick the right practices. For example, if the ability to adapt to changing needs is important, use multiple planning levels with feedback loops. For large complex space vehicles, we would have a five-year plan decomposed into an annual plan, decomposed into a quarterly plan, decomposed into a fortnightly plan, and finally decomposed into a daily plan. Each of these planning horizons will yield data that we need to use to further inform the next planning horizon.

For example, if my sprint plan is consistently only completing 75% of the planned backlog, it’s likely that the quarterly plan needs to be reduced or the staffing profile and tools adjusted. We use empirical data to adjust the plan.

You might wonder, with such a large plan, how does one maintain agility? The key to this form of planning is a regular cadence dedicated to updating the plan based on empirical data, with the fidelity of the plan decreasing with each planning horizon.