AI Prompts for Data Cleaning Scripts

Identify and describe common data quality issues in the dataset named [DATASET NAME]. Specifically, provide details on the following aspects: 1. Count and location of missing values in each column, 2. Total number of duplicate entries found, 3. Identification of any outliers or anomalies in numerical data, 4. Instances of inconsistent formats such as date formats and text casing, and 5. List of irrelevant data columns that do not contribute to the analysis. This comprehensive overview will assist in prioritizing your data cleaning efforts.

Analyze the data distribution for [COLUMN NAME] in [DATASET NAME]. Provide a summary that includes the mean, median, and mode; standard deviation and variance; quartiles and interquartile range; and frequency distribution. Additionally, create visualizations such as histograms or box plots to illustrate the data distribution. This comprehensive analysis will help identify potential data issues.

Draft a comprehensive data profiling report for the dataset named [DATASET NAME]. The report should include an overview of the dataset with the number of rows and columns, summary statistics for each column, data types and formats, percentages of missing values, and counts of unique values. Use this information to provide insights that will inform the data cleaning strategy.

Document the data sources and transformations for [DATASET NAME]. Include detailed information on the following aspects: 1. List the source names and types (e.g., CSV, API); 2. Describe the data collection methods used; 3. Outline the transformation steps taken to clean the data; 4. Note any assumptions made during the process; 5. Identify potential biases present in the data. This comprehensive documentation will enhance transparency and facilitate better understanding of your data cleaning efforts.

List the key stakeholders involved in ensuring data quality for [DATASET NAME]. For each stakeholder group, provide a brief description of their role: 1. Data owners - identify who is responsible for the data; 2. Analysts - specify who will utilize the data for analysis; 3. IT support - detail who manages the data infrastructure; 4. Compliance officers - explain who ensures adherence to data regulations; 5. End-users - clarify who relies on the data for decision-making. Engaging these stakeholders is essential for effective data cleaning.

Create a script in [PROGRAMMING LANGUAGE] that effectively handles missing values in [DATASET NAME]. Ensure the script includes methods for imputation such as mean, median, and mode, options for removing rows or columns with missing values, logging of all changes made during the process, visualizations of missing data patterns, and a testing phase using a sample dataset to verify functionality. This comprehensive approach will help maintain data integrity throughout the cleaning process.

Draft a script in [PROGRAMMING LANGUAGE] that identifies and removes duplicate entries from [DATASET NAME]. Ensure the script includes criteria for identifying duplicates, options for keeping either the first or last occurrence, logging of removed duplicates, testing procedures on a sample dataset, and a summary of the changes made to maintain data accuracy.

Create a script in [PROGRAMMING LANGUAGE] that converts data types in [DATASET NAME]. Ensure the script specifies which columns to convert and their corresponding target types. Include error handling mechanisms during the conversion process, log all conversion actions taken, and validate the output data types to ensure accuracy. Additionally, test the script on a sample dataset to confirm its functionality and effectiveness in standardizing your data.

Write a script in [PROGRAMMING LANGUAGE] to detect outliers in the [COLUMN NAME] of [DATASET NAME]. The script should include methods for detection such as Z-score and IQR, options for handling outliers like removal or transformation, and functionality for visualizing outliers. Additionally, implement logging for detected outliers and ensure the script is tested on a sample dataset to verify its effectiveness.

Draft a script in [PROGRAMMING LANGUAGE] that standardizes formats in [DATASET NAME]. Ensure the script specifies formats for dates, text, and numbers, handles inconsistencies in casing, logs all changes made, validates the output formats, and includes a testing phase on a sample dataset to confirm functionality. This comprehensive approach will enhance overall data consistency and reliability.

Create a comprehensive testing plan for the scripts developed for [DATASET NAME]. Include the following elements: 1. Define specific test cases for each script, detailing the purpose and scope of each test, 2. Specify the expected outcomes for each test case, 3. Identify appropriate sample datasets that will be used for testing, 4. Document the testing procedures step-by-step to ensure clarity, and 5. Schedule testing sessions, including timelines and responsible parties. This structured approach will ensure thorough validation of the scripts.

Conduct unit testing for the scripts created for [DATASET NAME]. Ensure to test each function individually, log any errors encountered, and validate outputs against expected results. After testing, adjust the scripts based on the results and document the entire testing process thoroughly. This approach will help identify and resolve issues early in the development cycle.

Conduct integration testing for the scripts in [DATASET NAME]. First, evaluate how the scripts interact with each other and identify any conflicts or issues that arise during execution. Next, validate the overall output of the combined scripts to ensure accuracy and consistency. Document any changes needed to improve functionality and schedule follow-up tests to verify that the issues have been resolved. This process will ensure that the scripts work cohesively and meet the required standards.

Validate the final outputs of your data-cleaning scripts for [DATASET NAME] against the original data requirements. Include the following steps: first, check for completeness and accuracy of the data; second, ensure that all data types are correct; third, review the outputs for any remaining data quality issues; fourth, document your findings in detail; and finally, prepare a summary report that outlines the effectiveness of your cleaning process.

Collect feedback from stakeholders regarding the cleaned data from [DATASET NAME]. First, schedule review sessions to discuss the data. Then, prepare a summary of the changes made to the dataset and gather input on its usability. Document any additional requirements that stakeholders may have and make adjustments to the data-cleaning scripts based on their feedback to ensure the data meets user needs effectively.

Draft a comprehensive data cleaning report for [DATASET NAME]. Ensure the report includes the following sections: 1. An overview of the initial data quality issues identified, 2. A detailed summary of the cleaning steps taken, 3. A final assessment of the data quality post-cleaning, 4. Visualizations that illustrate the before-and-after state of the data, and 5. Recommendations for improving future data management practices. This report should clearly communicate your efforts and findings.

Write detailed usage instructions for the scripts created for [DATASET NAME]. Include the following sections: 1. List the prerequisites needed to run the scripts, 2. Provide a step-by-step guide on how to execute the scripts, 3. Describe the expected outputs from running the scripts, 4. Outline common issues and their troubleshooting steps, and 5. Include contact information for support. Ensure the instructions are clear and easy to follow for users with varying levels of expertise.

Create a presentation summarizing the data cleaning process for [DATASET NAME]. Include the following sections: 1. Objectives of the cleaning project, 2. Key findings and challenges encountered during the process, 3. Visualizations that illustrate data quality improvements, 4. Feedback received from stakeholders, and 5. Recommended next steps for ongoing data management. Ensure the presentation is engaging and clear for your audience.

Share the cleaned dataset from [DATASET NAME] with stakeholders by following these steps: First, ensure the data is presented in a user-friendly format. Next, provide clear access instructions for stakeholders to retrieve the data. Highlight the key changes made during the cleaning process and summarize their significance. After sharing, gather feedback from stakeholders to understand their perspectives and document any follow-up actions that may be necessary. This structured approach will enhance collaboration and ensure effective communication.

Develop a comprehensive data quality monitoring plan for [DATASET NAME]. In your plan, define key metrics to track data quality, schedule regular checks to assess these metrics, and identify the responsible parties for monitoring. Additionally, document the procedures for addressing any new data quality issues that arise and outline a plan for periodic reviews to ensure ongoing data integrity.