< back to Expert Forum overview
The fifth BPM Expert Forum took place on December 7, 2021, with Mieke Jans and Angelique Koopman.
1: is process mining used in a more exploratory way (e.g. do you start from the data and just see where you end up), or do you use a pre-set set of things you check?
[Angelique] In our analysis we also use information about internal controls in the organisation’s process (e.g. thresholds in the system for 3-way match controls).
2: How do you audit a process via Process Mining if there are process steps which are not done in any system and, therefore, not having an event log? For example, SCRUM.
[Angelique] Process steps not done in the system are not included in the process mining application.
3: What do you mean with the term critical path in your audit procedure?
[Angelique] With the critical path analysis we evaluate the process variations for further investigation (using quantitative and qualitative criteria) and to identify the standard steps in the process.
4: Is currently the process mining primarily used in testing the controls related to financial statements? I wonder is it also possible to use process mining in understanding the business process, instead of only the financial related process. Perhaps, it is not very easy to be achieved by using event logs. What are you ideas?
[Angelique] We currently use it as a substantive analytical procedure; to understand the process and to evaluate the design of controls.
5: Using conformance, one discovers a great many deviations, more than in a ‘traditional’ audit. Should the auditor explain/address all these deviations? That would be a lot of work.
[Mieke] Indeed, using conformance checking leads to more deviations than what is feasible to inspect manually. So either the auditor falls back to sampling, or we look for a semi-automated way to classify all deviations as either exception or anomaly. This is where the auditor-in-the-loop mechanism comes in the picture.
6: The more auditing practice is relying on data analysis, the more important data provenance and quality becomes. Is the quality/provenance of data using in auditing considered in for example the ISA standards?
[Angelique] As auditors we always have to assess whether the information we use in our procedures is complete and accurate. So, when we apply data-analysis we validate the completeness and accuracy of the data extraction and data transformation before we use the data for our audit procedures.
7: Why do we find in practice so less control features (based on those process/data mining concepts) just implemented “ready to run” inside the existing ERP Systems (SAP, … )?
[Mieke] There are two separate parts to answer in this question. The one has to do with the technical feasibility, that there are different event logs that can be extracted out of one ERP system. Depending on which view is taken, different analyses can be run. So yes, in theory this could be added to an ERP system and it is safe to assume that ERP systems will do so in the (near?) future. Another aspect that this question touches upon, is the issue of keeping the different lines of defence separately. While certain controls are implemented in the system and active during the process enactment, process mining analyses are typically used afterwards as a separate line of defence, testing the effectiveness of the controls. So when integrating these functionalities in the ERP-system, it is important to keep the control architecture in mind as well.
8: Could/should all challenges be faced with a clear (audit) question upfront? e.g. choice between which items to choose as a case, which process model to choose, but also to be clear on which items can be a deviation?
[Mieke] Definitely this would help and will be a prerequisite to reach the highest quality. However, it won’t take away some core issues like the relational database structure that is often the start for an event log. In case you would like to have additional guidance, I am happy to point to a procedure on building the event log on my LinkedIn page (https://www.linkedin.com/in/mieke-jans-a832b62/detail/overlay-view/urn:li:fsd_profileTreasuryMedia:(ACoAAABmYYMBIO5-U1byGVYdDJ8e80b58MDhn7o,1635458880460)/).
9: My point here: every data analysis such as process mining should start with a clear question. Without a question you will never be able to perform a good analysis, and you will face the challenges you were talking about. Or at least, have more trouble coping them.
[Mieke] Indeed, this is completely correct. And being aware of them is key. Still, the challenges exist that it currently is very expensive to ‘take multiple pictures’. Clear advice on which views to take to answer which questions could help the profession.
Dr. Mieke Jans is an associate professor in Business Information Systems at Hasselt University, Belgium. She is active in the field of Accounting Information Systems, which is translated into a part-time appointment at the School of Business Economics, Maastricht University in the Accounting & Information Management department. Mieke earned her PhD in 2009 (at Hasselt University) with her research on the use of data mining and process mining techniques for the purpose of internal fraud risk reduction. From 2009 until 2014, Mieke combined working in industry with working in academics. Since September 2014, Mieke returned full-time to academics, where she now further researches how process mining can add value to the auditing profession. For this topic, she works closely together with industry and established researchers both in the accounting field as in the information systems field.
The financial auditor is charged with providing an opinion on the financial statements of organizations: are the statements representing a true and fair view of the organization? Traditionally, the auditor focuses on the transactions that are underlying the financial statements. More recently, auditors developed a keen interest in inspecting the process that outputs these transactions. If the process is under control, the transactions are reasonable to be complete, accurate and valid. Process mining assists the auditor in having insights into the level of control that organizations have over their processes.
Using process analytics in the context of an audit, raises the following questions:
Drs. Angelique Koopman RE RA has many years of experience in the audit profession. In different roles she applied a data driven audit and explored how to innovate the audit with technology. Currently, Angelique leads the Audit Innovation and Digital Transformation in the Netherlands. Recently, she also joined the EY Global innovation team to lead the EY Global Process Mining development and implementation.
In the past 8 years Angelique is passionate about the opportunities that Process Mining techniques can provide to improve audit quality and enhance the added value of the audit. It was and still is a bumpy road. Applying process analytics in the context of the financial statement audit provides interesting challenges. Nevertheless, Angelique is convinced that a close collaboration between academics and the audit industry will help to drive this audit innovation. Two specific challenges she will bring from the audit industry to discuss with you during the BPM Expert Forum meeting are: