Robotic process automation for PG&E
Robotic process automation for PG&E
The project aimed to automate the data retrieval process from PG&E, a utility company, by developing a bot using Robomotion. The bot performs a series of tasks including logging in to the PG&E website, downloading PDF files containing essential data, extracting relevant information, storing it in a database, and uploading the PDF files to both Google Drive and Digital Ocean Space. Additionally, it sends notifications to Discord at the start and resumption of the bot, and also in case of any errors occurring during the process.
Tasks Completed:
- Init Config:
Configured initial settings to ensure smooth operation of the bot.
- Connect to Database:
Established a connection to the database for storing extracted data.
- Excel Operations:
Implemented functionality to access Excel files containing necessary data.
- Login to Website & Get Accounts:
Automated login to the PG&E website and retrieval of account information.
- Download & Read PDF:
Enabled the bot to download PDF files associated with each account and read their contents.
- Perform OCR using Tesseract OCR:
Utilized Tesseract OCR for Optical Character Recognition on PDFs, extracting text data.
- Get Values From PDF & Prepare Data:
Extracted relevant information from PDF files and formatted it for database storage.
- Insert to Database:
Inserted the prepared data into the database for future reference and analysis.
- Upload PDFs to Google Drive and Digital Ocean Space:
Automated the process of uploading PDF files to both Google Drive and Digital Ocean Space for backup and accessibility.
- Sending Notification to Discord:
Implemented functionality to send notifications to Discord at the start and resumption of the bot, and also in case of any errors occurring during the process.
- Integrations: PostgreSQL, Excel, Tesseract OCR, Google Drive, Digital Ocean Space, Discord, PDF, Image Processing
Robotic process automation for SfWater
Robotic process automation for SfWater
The project aimed to automate the data retrieval process from SfWater, a utility company, by developing a bot using Robomotion. The bot performs a series of tasks including logging in to the SfWater website, downloading PDF files containing essential data, extracting relevant information, storing it in a database, and uploading the PDF files to both Google Drive and Digital Ocean Space. Additionally, it sends notifications to Discord at the start and resumption of the bot, and also in case of any errors occurring during the process.
Tasks Completed:
- Init Config:
Configured initial settings to ensure smooth operation of the bot.
- Connect to Database:
Established a connection to the database for storing extracted data.
- Excel Operations:
Implemented functionality to access Excel files containing necessary data.
- Login to Website & Get Accounts:
Automated login to the PG&E website and retrieval of account information.
- Download & Read PDF:
Enabled the bot to download PDF files associated with each account and read their contents.
- Perform OCR using Tesseract OCR:
Utilized Tesseract OCR for Optical Character Recognition on PDFs, extracting text data.
- Get Values From PDF & Prepare Data:
Extracted relevant information from PDF files and formatted it for database storage.
- Insert to Database:
Inserted the prepared data into the database for future reference and analysis.
- Upload PDFs to Google Drive and Digital Ocean Space:
Automated the process of uploading PDF files to both Google Drive and Digital Ocean Space for backup and accessibility.
- Sending Notification to Discord:
Implemented functionality to send notifications to Discord at the start and resumption of the bot, and also in case of any errors occurring during the process.
- Integrations: PostgreSQL, Excel, Tesseract OCR, Google Drive, Digital Ocean Space, Discord, PDF, Image Processing
Auto Submit Bills on Appfolio.com
Auto Submit Bills on Appfolio.com
This automation project simplifies the process of submitting bills on Appfolio.com for real estate administrators. By automating the steps involved, users can quickly log in, add new bills, fill in necessary details, upload files, save bills, and seamlessly move on to adding more bills without interruptions.
𝐊𝐞𝐲 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐬:
- Easy Navigation: The automation script swiftly takes users to the bill submission section after logging in with real estate admin credentials.
- Automatic Login: Users are automatically logged in with their admin details, ensuring hassle-free access to the bill management interface.
- Simplified Bill Submission: The script fills in essential fields like vendor info and invoice details.
- File Upload Made Easy: It automates file uploads, saving users from manually selecting and uploading files like invoices or receipts.
- Error Handling: The script detects and handles errors during submission, ensuring reliable bill processing.
- Efficient Bill Saving: Once all details are entered and files uploaded, bills are automatically saved securely.
- Add Another Bill: After submitting a bill, users can seamlessly start adding another, optimizing workflow.
- Scalability: The solution can manage bills across multiple properties or portfolios, scaling as needed.
- RPA Tool: Robomotion RPA
- Language: Javascript
Dynamic E-commerce Data Extraction Tool
Dynamic E-commerce Data Extraction Tool
A web scraping tool designed to extract product information from an e-commerce website. The tool utilizes JavaScript to scrape various data elements such as product titles, prices, descriptions, specifications, images, categories, and URLs from the target website's HTML structure. It organizes this data into a structured JSON format and performs actions such as opening product URLs in new browser tabs and sending the collected data to a designated endpoint via a POST request. The project aims to provide a comprehensive solution for gathering and processing product data from e-commerce websites efficiently.
𝐊𝐞𝐲 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐬:
- Dynamic Data Extraction: Extracts diverse product information like titles, prices, descriptions, specifications, images, categories, and URLs from webpage HTML, employing DOM manipulation techniques.
- Data Transformation: Transforms extracted data into structured JSON format, facilitating manipulation and transmission, with options for modifying data elements as needed.
- Parallel Processing: Opens product URLs in separate browser tabs for simultaneous processing, improving efficiency by enabling concurrent scraping of multiple pages.
- Pagination Handling: Includes logic for navigating pagination, allowing scraping of data from multiple search result pages, enhancing scalability for large datasets.
- Asynchronous Execution: Delays execution using setTimeout, ensuring controlled and asynchronous processing, preventing performance issues and browser crashes.
- Error Handling: Incorporates checks to manage variations in HTML structure and filter out incomplete or irrelevant data, ensuring integrity and accuracy of collected information.
- Scalability: Designed to handle large volumes of product data across multiple pages, offering flexibility for customization and integration with other systems.
- Automation: Automates scraping process, reducing manual effort and enabling continuous data collection for regular updates, maintaining data currency and relevance.
- Language: Javascript
Villicus Monitoring System Server using Python
Villicus Monitoring System Server using Python
The Villicus Monitoring System Server is a robust TCP server application tailored for industrial automation and control systems. Its primary function is to facilitate seamless communication between client devices and Programmable Logic Controllers (PLCs), with a specific focus on the Micro800
𝐊𝐞𝐲 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐬:
- Real-time Communication: The server establishes and maintains TCP/IP connections with client devices, ensuring real-time data transmission and control signals exchange over IP networks.
- Efficient Data Serialization: Utilizing JSON (JavaScript Object Notation) serialization, the server efficiently structures data for transmission, enhancing compatibility and ease of integration with client applications.
- Data Compression: Advanced data compression techniques, including zlib and base64 encoding, are employed to optimize bandwidth usage and maximize network performance, crucial for handling large volumes of data in industrial environments.
- PLC Interaction: The server seamlessly interacts with Micro800 series PLCs, offering essential functionalities such as reading tag values, writing tag values, and retrieving tag updates. This enables clients to remotely monitor, diagnose, and control PLC operations.
- Concurrent Client Handling: Leveraging multithreading capabilities, the server efficiently handles multiple client connections concurrently, ensuring scalability and responsiveness to accommodate a high number of client devices without compromising performance.
- Fault Tolerance: Robust error handling mechanisms are integrated into the server to detect and gracefully manage exceptions, ensuring uninterrupted operation and reliability under diverse operating conditions.
- Configurability: The server's configuration parameters, including IP address and port number, can be easily customized via an external configuration file (app_config.py), providing flexibility for deployment in various industrial network environments.
This project is designed for practical deployment in real-world industrial settings, offering a reliable and scalable solution for facilitating communication between client devices and PLCs.
- Language: Python