Skip to content

Commit

Permalink
[KYUUBI #6765] [CI] speed up the script for freeing disk space by bat…
Browse files Browse the repository at this point in the history
…ching removals

# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- batching removals to avoid repeatedly reading and updating the apt-get databases
- 50% speed up the execution time for freeing disk space from 2 minutes or more , to about 1 minutues.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6765 from bowenliang123/speedup-freedisk.

Closes #6765

dd475c2 [Bowen Liang] update (+1 squashed commit) Squashed commits: [688793f] update

Authored-by: Bowen Liang <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
  • Loading branch information
bowenliang123 authored and yaooqinn committed Nov 14, 2024
1 parent 2453efe commit e03207b
Showing 1 changed file with 31 additions and 23 deletions.
54 changes: 31 additions & 23 deletions .github/scripts/free_disk_space.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

# This script is inspired by Apache Spark

set -x

echo "=================================="
echo "Free up disk space on CI system"
echo "=================================="
Expand All @@ -28,28 +30,34 @@ dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n | tail -n 100
df -h

echo "Removing large packages"
sudo rm -rf /usr/share/dotnet/
sudo rm -rf /usr/share/php/
sudo rm -rf /usr/local/graalvm/
sudo rm -rf /usr/local/.ghcup/
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/local/lib/node_modules

sudo rm -rf /opt/az
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /opt/hostedtoolcache/go
sudo rm -rf /opt/hostedtoolcache/node

sudo apt-get remove --purge -y '^aspnet.*'
sudo apt-get remove --purge -y '^dotnet-.*'
sudo apt-get remove --purge -y '^llvm-.*'
sudo apt-get remove --purge -y 'php.*'
sudo apt-get remove --purge -y '^temurin-\d{n,}.*'
sudo apt-get remove --purge -y snapd google-chrome-stable microsoft-edge-stable firefox
sudo apt-get remove --purge -y azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri
sudo apt-get autoremove --purge -y
sudo apt-get clean
sudo rm -rf /usr/share/dotnet/ \
/usr/share/php/ \
/usr/local/graalvm/ \
/usr/local/.ghcup/ \
/usr/local/share/powershell \
/usr/local/share/chromium \
/usr/local/lib/android \
/usr/local/lib/node_modules \
/opt/az \
/opt/hostedtoolcache/CodeQL \
/opt/hostedtoolcache/go \
/opt/hostedtoolcache/node

df -h

sudo apt-get remove --purge -y \
'^aspnet.*' \
'^dotnet-.*' \
'^llvm.*' \
'php.*' \
'^temurin-\d{n,}.*' \
snapd google-chrome-stable microsoft-edge-stable firefox \
azure-cli google-cloud-sdk mono-devel msbuild powershell libgl1-mesa-dri

df -h

sudo apt-get autoremove --purge -y \
&& sudo apt-get clean \
&& sudo apt-get autoclean

df -h

0 comments on commit e03207b

Please sign in to comment.