Recovering Source Code from Google App Engine

Introduction

The deprecation of certain Google App Engine tools has made it challenging to download source code directly from the App Engine. However, by leveraging the Google Cloud Storage bucket named "artifacts," it is still possible to recover the source code of your App Engine applications. In this tutorial, we will walk through the steps to download and extract the source code from the artifacts bucket.

Prerequisites

• Access to the Google Cloud Console with appropriate permissions.
• Basic knowledge of the Google Cloud Storage and Google Cloud SDK.

Step 1: Download the Artifacts Bucket

1. Open the Google Cloud Console and navigate to your project.
2. Locate the "artifacts" bucket associated with your project.
3. Use the gsutil command to download the entire contents of the artifacts bucket:
   gsutil -m cp -r gs://bucketname local-directory
   Replace "bucketname" with the name of your artifacts bucket, and "local-directory" with the directory path on your local machine where you want to download the files.

Step 2: Navigate to bucketname/containers/images

4. Navigate to the downloaded directory containing the artifacts.
5. Navigate to bucketname/containers/images within the downloaded artifacts directory.
   (Replace "bucketname" with the actual name of your artifacts bucket.)

Step 3: Batch Rename the Files

6. Within the bucketname/containers/images directory, rename the files to replace ":" with "-" using the following command:
   find . -type f -exec rename 's/:/-/' {} \;

Step 4: Rename Gzipped Files

7. Make sure you have the rename command available (if not, you can install it).
8. Run the following command to rename all gzipped files to include the ".tar.gz" extension:
   for file in *; do if file "$file" | grep -q "gzip compressed"; then mv "$file" "$file.tar.gz"; fi; done

Step 5: Extract Tarballs Containing /srv/ or /workspace/

9. Create a target directory for the extracted source code, e.g., "apps/":
   mkdir apps
10. Run the script to extract tarballs that contain either the "/srv/" or "/workspace/" directories to the "apps/" directory:
    find . -type f -name "*.tar.gz" -exec sh -c 'if ! [ -d "$2/$(basename "$1" .tar.gz)" ] && (tar tf "$1" | grep -q "/srv/" || tar tf "$1" | grep -q "/workspace/"); then mkdir -p "apps/$(basename "$1" .tar.gz)"; tar xf "$1" -C "apps/$(basename "$1" .tar.gz)"; fi' sh {} "apps" \;

Conclusion

By following these steps, you can recover the source code of your Google App Engine applications. The extracted files will be available in the "apps/" directory, containing all the deployed versions that include either the "/srv/" or "/workspace/" directories. Take advantage of the correct date information to identify the most recent versions and examine the files to find the exact deployment you need.

Note: While this process helps recover the code, manual effort may still be required to identify the latest running version and locate the exact deployment that may be in production.

Remember, the scripts provided in this tutorial were created with the assistance of ChatGPT, an AI language model, to help streamline the recovery process.