A scraper that only lives in memory loses its crawl state, dedupe set, and downloaded pages the moment the process restarts. DenkOps gives every slot a durable /persist disk mounted at a fixed path, so a Flask-triggered scraper can checkpoint URLs already visited and resume a long crawl exactly where it left off.
import requests
from bs4 import BeautifulSoup
from flask import Flask, jsonify
app = Flask(__name__)
SEEN = open("/persist/seen.txt", "a+")
@app.post("/scrape")
def scrape():
r = requests.get("https://example.com/listings", timeout=10)
soup = BeautifulSoup(r.text, "html.parser")
items = [a["href"] for a in soup.select("a.listing")]
SEEN.write("\n".join(items) + "\n")
SEEN.flush()
return jsonify(count=len(items))Deploy it: install the plugin, say "deploy on DenkOps", and get a live SSL URL.
Start on DenkOps →Write to /persist, it's a durable disk attached to your slot that survives restarts and redeploys, unlike a serverless function's ephemeral filesystem.
Yes, through the zero-trust egress gateway, outbound calls are allowed and logged, so your scraper can reach any target site.