AI Implementation Part 5: The requirements spec doesn’t exist

AI now takes on work that software never could, such as summarising a customer call, handling a complaint or responding to a customer’s request for a quote. These tasks rely on judgement. Nobody can write down upfront what a really good answer looks like, yet an experienced expert can look at an output and say in 90 seconds what is missing.

This session explores why AI applications must be tested differently from traditional software. The same question can give different answers, and every change to a model, prompt or data source needs far more test cases than engineers are used to. The harder problem is deciding what to test for when experts can’t describe it and often disagree.

You will learn how to build the loop that turns expert judgement into something you can engineer against: generate outputs, let experts react, and capture every reaction as a test case. Over time the eval set becomes the only honest requirements spec you will get. Cases from banking, legal AI and large tech companies show how it works in practice.

The session closes on a practice engineers already know: developers learn good code in code review, and domain experts need the same.

Key takeaways

• Why judgement-based AI tasks have no requirements spec, and what to do instead
• How many test cases you actually need when the answer varies
• How to get experts reacting to outputs instead of writing criteria
• What to do when experts disagree about what good looks like
• How to let business experts experiment without slowing down development
• How to turn expert feedback into an eval set that runs on every change

Target audience

• Software engineers and developers building with AI
• AI and ML engineers
• Product managers and product owners
• Tech leads and architects
• CTOs and technology leaders
• Domain experts who work with AI teams

Speakers: 

Louise Mehl is a partner at Implement with a strong background as a Data Protection Officer (DPO) and AI implementation lead. Louise is an expert in GDPR, DORA and AI Act compliance and has led several major GRC and AI governance programmes for financial institutions.

Rasmus Risager Lindegaard is a partner at Implement and heads AI in Financial Services. He was Head of AI at Lunar and founded Avido, which built tools for testing and evaluating AI applications for regulated companies.

-----------------

Registration procedure for the webinar:

Once you have registered you will receive a confirmation e-mail. At the bottom of this mail you will find a link to the webinar. You log in through this link and are forwarded to the site where the webinar is shown.

Should you have any problems logging in, you are welcome to contact us at: webinarsupport@ida.dk

Also check out our FAQ site where you can get answers to the most frequently asked questions in connection with webinars: FAQ about IDA webinars

Privacy info: If you would like to participate anonymously, we recommend you join by browser instead of using the Zoom software client or app, and when Zoom asks you to provide your name you can enter "Anon" or another pseudonym. Zoom processes personal information in global datacenters including USA and processing is carried out according to EU Commission standard data protection clauses. Privacy Shield. For further details, we refer you to the Zoom privacy policy available here: https://zoom.us/privacy

It is not allowed to share the zoom link or the video link with others, or to record audio and images of the webinar.

Sign up: Not a member? In order to sign up, you must create a user profile. Note, when creating a user profile, you will be able to access IDA's online services - you will not become a member of IDA.

Important information regarding personal data on participant lists: Your name, title and company name will appear on the participant list for this event if you have previously given consent or have given consent in connection with registration to this event. The participant list is available to the organizer and the other participants at the event itself in the form of an online participant list. At some events, a name tag is also given out, where your name, title and company name appear. If you unsubscribe from the event, your information will no longer appear on the participant list.

You can read more about consent here.