Workflows for Online Data Collection

Configure your computer

Statistics and Computation

Set up TeXmacs ^{1 mins} Set up Visual Studio Code ^{3 mins} Set up Sublime Text ^{2 mins} Install Python Packages ^{1 mins} Install Stata Packages ^{1 mins} Set up Git and GitHub ^{1 mins} Set up LaTeX ^{2 mins} Set up MATLAB ^{1 mins} Set up Pandoc ^{1 mins} Set up Perl ^{1 mins} Set up Python ^{1 mins} Set up R and RStudio ^{4 mins} Set up Stata ^{2 mins}

Automation and Workflows

Making Path Settings Permanent ^{2 mins} Set up Command Line Tools ^{3 mins} Set up Docker ^{4 mins} Set up Make ^{2 mins} Set up Snakemake ^{1 mins} Set up TeamViewer ^{1 mins} Set up the AWS Client ^{1 mins}

Research Infrastructure

Learn About the Various Ways to Setup a Research Infrastructure ^{5 mins} Cloud Computing with SURFsara's Cartesius Cluster ^{1 mins} Cloud Computing with SURFsara's LISA Cluster ^{1 mins} Getting Started with Research Cloud ^{3 mins} HPC TiU for Cloud Computing ^{2 mins} Set up SURF Research Drive and Rclone to Store Data ^{2 mins} Setting Up a New Workspace, Configuring Git and Adding Research Drive on Research Cloud ^{2 mins} Use Google Colab to Run Python in the Cloud ^{1 mins}

Task-specific Configurations

Configure Python virtual environments ^{3 mins} Configuring Python for Web Scraping ^{3 mins}

Develop your research skills

Learn to Code

Bring AI to your command line/terminal ^{6 mins} Learn Bash Commands ^{4 mins} Learn Python and Julia ^{1 mins} Learn R ^{2 mins} Learn Regular Expressions ^{11 mins}

Tips

Searching for Literature on the Web ^{2 mins} Build and Maintain a Reference List ^{5 mins} Cite Efficiently in Your Academic Paper ^{4 mins} Principles for Good Coding ^{2 mins} Python Coding Style Guidelines ^{1 mins} R Coding Style Guidelines ^{2 mins} Stata Coding Style Guidelines ^{4 mins} Comparing text editors ^{5 mins}

Collect your data

Web Scraping & APIs

Avoid Getting Blocked While Scraping ^{1 mins} Extract Data From APIs ^{3 mins} Read & Write Data From APIs ^{1 mins} Scrape Dynamic Websites ^{2 mins} Scrape Static Websites ^{4 mins}

Workflows for Online Data Collection

Assess what Data is Available ^{4 mins} Calculate Sample Sizes for Web Scrapers ^{1 mins} Monitor and Safeguard Your Data Quality ^{3 mins} Online Data Collection and Management ^{1 mins} Online Data Sources for Economists ^{1 mins} Safeguard Legal Compliance When Scraping ^{6 mins}

Store and document your data

Store Data

Choose a Data Repository for Long-Term Storage ^{4 mins} Download Data Programmatically ^{6 mins} Safely Store Your Credentials By Configuring Environment Variables ^{2 mins} Use AWS S3 Buckets ^{3 mins}

Document Data

Document New Data ^{10 mins} README Best Practices ^{4 mins}

Prepare and visualize your data

Data Preparation

Handle Large Datasets In Python ^{3 mins} Import Large Datasets Into R ^{4 mins} Practical Example with Dask ^{6 mins} Clean Numeric Data Scraped From Social Media ^{1 mins} SPSS Data in R ^{9 mins} Data Preparation and Workflow Management ^{1 mins} Text Pre-processing in R ^{7 mins}

Data Visualization

Data Visualization Best Practices ^{9 mins} Plotting with Matplotlib and Seaborn in Python ^{4 mins}

Analyze data

Regressions

Run a Regression Analysis ^{3 mins} Easy Standard Error Adjustment with `estimatr` ^{3 mins} Evaluate Model Assumptions ^{2 mins} Compute Marginal Effects ^{6 mins} Doing Calculations with Regression Coefficients Using deltaMethod ^{2 mins} Generate regression tables in R with the `modelsummary` package ^{17 mins} Intro to IV Estimation ^{5 mins} Create a publication-ready LaTeX regression table with `kableExtra` in R ^{8 mins} Generate a coefficient plot in R with the `modelplot` function ^{6 mins} Interpreting Regression Models in R ^{10 mins} Mediation Analysis Tutorial in R ^{12 mins} Interaction Terms ^{15 mins}

Regression Discontinuity

Sharp RD Designs ^{6 mins} Regression Discontinuity Plots ^{7 mins} Continuity-Based Approach ^{5 mins} Fuzzy RD Designs ^{7 mins} Fuzzy RDD in Practice ^{3 mins} Local Randomization Approach ^{9 mins} Validation and Falsification Analysis for RDD ^{9 mins} Impact evaluation with Difference-in-Differences and Regression Discontinuity ^{4 mins} Synthetic Control for Impact Evaluation ^{4 mins}

Regressions With Panel Data

An Introduction to Panel Data ^{11 mins} Fixed Effects Regression Assumptions ^{4 mins} The Pooled OLS Model ^{2 mins} The First-Difference Estimator ^{2 mins} The Fixed Effects Model (Within Estimator) ^{5 mins} Fixed-Effects Estimation in R With the `fixest` Package ^{7 mins} The Between Estimator ^{1 mins} The Random Effects Model ^{6 mins} An Introduction to Difference-in-Difference Analysis ^{9 mins} Canonical Difference-in-Difference as a Regression ^{7 mins} Staggered Difference-in-Difference Estimation ^{8 mins}

Machine Learning

Extreme gradient boosting with XGBoost in R ^{11 mins} In-depth Introduction to Machine Learning and R ^{1 mins}

Automate and execute your work

Automate your Workflow

Use Makefiles to Manage, Automate, and Reproduce Projects ^{4 mins} Configure a Self-hosted runner for GitHub Actions workflows ^{8 mins} Implement a GitHub Actions workflow for code formatting and testing ^{5 mins} Enhance Reproducibility and Automation with GitHub Actions ^{5 mins} Debugging Makefiles With Remake ^{6 mins} How to Automatically Install R Packages Used in a Project ^{1 mins} Schedule Recurring Tasks (e.g., by day, hour) ^{5 mins} Error Handling in Stata Workflows Using R ^{3 mins} tableFill for Reproducible Tables ^{3 mins} Make Tables and Figures Reproducible with Autofilling Values ^{1 mins} Share Large Temporary Files ^{3 mins} Use this Checklist to Improve Your Project's Structure and Efficiency ^{3 mins}

Make your Work Reproducible

Use Software Environments to Ensure Replicability ^{6 mins} Configure a VM with GPUs in Google Cloud ^{7 mins} Package Management for R: renv ^{3 mins} Import and run a project environment with Docker on Google Cloud ^{11 mins} Export a Python Environment with Docker and Share It Through Docker Hub ^{11 mins} Docker for Reproducible Research ^{7 mins} Manage your R Packages with Packrat ^{5 mins} Run RStudio on AWS using Docker ^{5 mins} Monitor and solve memory constraints in your computational environment ^{4 mins} Handle storage within your Google Cloud instance ^{3 mins}

Collaborate and share your work

Write your Paper

Forest Plot Generation in R ^{5 mins} Visualizing data with Plotnine and Altair ^{2 mins} Use Overleaf to Collaborate on Research Papers ^{1 mins} LyX for Professional Research Papers and Presentations ^{1 mins} Get Started With Our Bookdown Thesis Template ^{1 mins} Get Started With Our LaTeX Thesis Template ^{1 mins} Stata Graphs Made Easy ^{2 mins} Stargazer: Exporting Publication-Ready Tables in R ^{1 mins} Cheatsheet for LaTeX Math Commands ^{6 mins}

Professionalize Project Management

Manage Your Projects Using the GitHub Project Board ^{1 mins} How to Prepare for (and Follow-up On) Scrum Meetings ^{2 mins} Best Practices for GitHub Issues Management ^{8 mins} Bring Open Science Practices to your Research Group ^{1 mins}

Publish on the Web

Build Interactive Dashboards with R Shiny ^{5 mins} Publish Ebooks and Interactive Articles Directly From R ^{1 mins}

Building Blocks

Configure your computer

Statistics and Computation

Automation and Workflows

Research Infrastructure

Task-specific Configurations

Develop your research skills

Learn to Code

Tips

Collect your data

Web Scraping & APIs

Workflows for Online Data Collection

Store and document your data

Store Data

Document Data

Prepare and visualize your data

Data Preparation

Data Visualization

Analyze data

Regressions

Regression Discontinuity

Regressions With Panel Data

Machine Learning

Automate and execute your work

Automate your Workflow

Make your Work Reproducible

Collaborate and share your work

Version and Share Code with Github

Write your Paper

Professionalize Project Management

Publish on the Web

Google Analytics (functional)

Google Tag Manager (functional)