Automation with Machine Learning (ML) in DevOps, RPA and BPA

Software Automation

Using software to perform tasks that used to be done manually.

Benefits

• Efficiency
• Reduce errors
• Free up human workers to focus on more creative/strategic/complex tasks

Machine Learning (ML)

ML is a branch of AI that focuses on developing algorithms and models - enables computers to learn from data and make decisions or predictions without being explicitly programmed for every single task.

Key Principles

• Artificial intelligence is the goal. Machine learning is one of the means to reach that goal
• Teaching machines to recognise patterns and improve based on experience

Model

The output of a machine learning training process. It represents what the system has learned from data using an algorithm.

Agent

Used primarily in reinforcement learning, an agent is the entity that takes actions in an environment to achieve a goal. It learns by receiving rewards or penalties.

Summary

Machine learning algorithms are mathematical procedures that the computer uses to learn from data, by finding patterns and relationships and using this information to make predictions or decisions. After training, the model is tested with new data to evaluate its performance and ability to generalise its learning to unseen data.

How ML Supports Automation

ML supports automation across three key frameworks:

DevOps

• Predictive analysis for deployment fails
• Automates monitoring/log analysis
• Optimises CI/CD pipelines
• Example: ML models detect unusual error patterns during deployment

RPA

• Handles unstructured data
• Supports intelligent document processing
• Bots can learn from variations in workflows
• Example: Automating invoice approvals, chatbots handling customer service queries

BPA

• Process optimisation by analysing workflows
• Predicts bottlenecks and suggests improvements
• Automates decision making
• Example: Supply chain forecasting and inventory optimisation

DevOps (Developer Operations)

A set of practices intended to reduce the time between committing a change to a system, and the change being placed into production - improve collaboration and communication between software development and IT operations.

Goals and Emphasis

• Goal: shorten development life cycle, increase frequency of releases, improve quality and reliability
• Emphasis on automating repetitive tasks (code testing, integration, deployment, monitoring)
• Experimentation - learning from mistakes and adapting based on feedback

Core Practices

• Continuous integration (CI): automatically builds and tests code on every commit
• Continuous delivery (CD): automatically prepares and deploys tested code to production environments
• Infrastructure as Code (IaC): managing and provisioning infrastructure using code instead of manual configuration
• Monitoring and logging: track performance and detect issues with tools
• Automated testing: run test automatically to ensure quality and protection

ML in DevOps

• Detects anomalies in system behaviour
• Predicts system failures before they occur
• Optimises build/test/deploy pipelines

Robotic Process Automation (RPA)

Using software bots to automate repetitive, rule-based tasks (mimic human actions in digital systems).

Overview

• Can interact with applications, manipulate data, trigger responses and communicate with other systems
• Tasks don't need decision making or human judgement
• Replaces tasks that are repetitive, rule based, high volume, manual, time consuming
• Typical tasks: read data from spreadsheets/databases, data validation/processing, enter data into systems, generate reports, send notifications
• Example: automating data entry into databases, extracting information from invoices, automating responses to common customer inquiries

ML in RPA

• Allows bots to make decisions or adapt when tasks aren't exactly the same
• Adds intelligence to bots

Summary

RPA automates routine and repetitive tasks performed by humans on computers, improving efficiency, reducing errors and allowing employees to focus on more complex and valuable work. It helps streamline operations and improve productivity for businesses.

Business Process Automation (BPA)

Automating entire business workflows (not just single tasks) - complex, multi-step business processes that traditionally require human input, coordination and oversight.

Overview

• Arranging multiple tasks, systems, data sources, and people through automated workflows
• Purpose is to streamline operations, improve efficiency, reduce manual effort
• Example: automating customer onboarding process

ML in BPA

• Analyses process performance and recommends improvements
• Predicts workflow bottlenecks
• Automates decision making in processes

Summary

BPA uses technology to streamline and automate complex business processes to improve efficiency, reduce costs and improve accuracy. Human resources can be used for more strategic tasks and respond more quickly to changes.

MLOps

The automated process of designing, training and deploying machine learning models (using many of the same principles as DevOps). Brings together teams developing ML models and operational teams who deploy and support the models in production.

Design Phase

Transforms a general business goal into a technically-viable ML problem. It also ensures the solution aligns with user needs and measurable success criteria.

• Defining the business problem to be solved: Identify a clear, actionable goal that delivers value to the organisation
• Refactoring the business problem into a machine learning problem: Translate business needs into a technical question; Choose appropriate ML task type: classification, regression, clustering, recommendation
• Defining success metrics: Set quantitative measures of performance based on business goal; Define business success
• Researching available data: Identify what datasets are accessible internally/externally; Assess volume, variety, velocity, veracity of the data; Determine data relevance, quality, ethical/legal considerations

Model Development Phase

• Data wrangling: Clean, transform and prepare raw data for analysis (e.g. missing values, remove outliers, convert formats)
• Feature engineering: Create meaningful input variables from raw data; select features that improve model accuracy and relevance
• Model training: Use prepared data to train the ML model
• Model testing and validation: Assess the model's accuracy and generalisability; Evaluate accuracy, precision, recall, MSE

Operations Phase

• Model deployment: Integrate trained model into production environment; Use automated pipelines
• Supporting operations/use: Ensure it is accessible and usable by end-users or other systems; Documentation, integration with front end, etc
• Monitoring model performance: Continuously track accuracy and latency; Detect issues and monitor logs

MLOps vs DevOps

• Scope: DevOps focuses on the software development life cycle; MLOps focuses on the ML life cycle
• Complexity: ML models are often more complex than traditional software applications, and require specialised tools and techniques for development and deployment
• Data: ML models rely on data for training and inference, meaning additional challenges for managing and processing data
• Regulation: ML models may be subject to regulatory requirements, which impacts the development and deployment process

Summary

DevOps involves reducing barriers between development and operations teams to create a more efficient, collaborative and automated process for building, testing and deploying software. Organisations can deliver higher-quality software faster and respond better to feedback.