Skip to content

Selenium.md

Latest commit

 

History

History
180 lines (132 loc) · 7.14 KB

Selenium.md

File metadata and controls

180 lines (132 loc) · 7.14 KB
Raw

Selenium

Browser-automation library that drives a real browser (Chrome, Firefox) via WebDriver — used to test and scrape JavaScript-heavy web applications that a plain HTTP client cannot render.

Overview

Selenium controls an actual browser, executing JavaScript, following redirects, and interacting with the DOM the way a user would. Where requests sees only the initial HTML, Selenium sees the fully-rendered page. In security work this makes it ideal for testing single-page applications, automating logins with client-side crypto, capturing screenshots of large host lists, and reproducing complex user flows during authorized assessments.

Installation

pip install selenium webdriver-manager

Note

Modern Selenium (4.6+) auto-manages drivers via Selenium Manager, so a system Chrome/Firefox is usually enough. webdriver-manager is a fallback for pinning driver versions.

Basic Usage

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless=new")

driver = webdriver.Chrome(options=options)
try:
    driver.get("http://127.0.0.1:8000")
    print(driver.title)
    print(driver.find_element(By.TAG_NAME, "h1").text)
finally:
    driver.quit()          # always quit, or the browser process leaks

Selenium drives a real browser, so JavaScript executes and the DOM you see is the rendered one.

Important APIs

API Purpose
webdriver.Chrome/Firefox(options=) Start a browser session
driver.get(url) Navigate
driver.find_element(By.X, value) First match; raises NoSuchElementException
driver.find_elements(By.X, value) All matches; returns [] when none
By.ID / CSS_SELECTOR / XPATH / TAG_NAME Locator strategies
element.text / .get_attribute(name) Read content
element.click() / .send_keys(text) Interact
WebDriverWait(driver, n).until(cond) Explicit wait — the correct way to wait
driver.page_source Rendered HTML, ready for [[BeautifulSoup]]
driver.get_cookies() Session cookies
driver.quit() Close the browser and free the process

Selenium 4 manages drivers automatically via Selenium Manager; older versions needed a manually installed chromedriver.

Example

Headless page load and title extraction:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

opts = Options()
opts.add_argument("--headless=new")
opts.add_argument("--no-sandbox")

driver = webdriver.Chrome(options=opts)
driver.get("https://example.com")
print("Title:", driver.title)
driver.quit()

Output

Title: Example Domain

Automate a login form and read the resulting page:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options

opts = Options()
opts.add_argument("--headless=new")
driver = webdriver.Chrome(options=opts)

driver.get("https://practice.example.lab/login")   # authorized lab target
driver.find_element(By.NAME, "username").send_keys("tester")
driver.find_element(By.NAME, "password").send_keys("Passw0rd!")
driver.find_element(By.CSS_SELECTOR, "button[type=submit]").click()

print("Logged-in URL:", driver.current_url)
driver.quit()

Output

Logged-in URL: https://practice.example.lab/dashboard

Bulk screenshot a list of authorized hosts (recon evidence gathering):

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

opts = Options()
opts.add_argument("--headless=new")
driver = webdriver.Chrome(options=opts)

hosts = ["https://example.com", "https://example.org"]
for i, url in enumerate(hosts):
    driver.get(url)
    driver.save_screenshot(f"shot_{i}.png")
    print("captured", url)

driver.quit()

Output

captured https://example.com
captured https://example.org

Security Use Cases

  • Testing SPA / JS-heavy apps — reach functionality that only appears after client-side rendering during an authorized assessment.
  • Automated login flows — script MFA-free logins, session setup, and multi-step workflows to reach deeper application state.
  • Screenshotting at scale — capture a large in-scope host list to triage interesting web apps quickly (an EyeWitness-style workflow).
  • Client-side vuln reproduction — trigger and observe DOM-based XSS or client-side redirects in a real browser.
  • UI-driven regression checks — confirm that security fixes (CSP, cookie flags) behave correctly in-browser.

Common Mistakes

  • Forgetting driver.quit() — orphaned browser processes accumulate and exhaust memory. Use try/finally or a context manager.
  • Using time.sleep() instead of WebDriverWait — either flaky or needlessly slow.
  • Not handling NoSuchElementException — find_element raises; find_elements returns an empty list.
  • Brittle absolute XPaths that break on any markup change; prefer IDs or stable CSS selectors.
  • Interacting with a stale element after the page re-renders, raising StaleElementReferenceException.
  • Assuming a driver binary must be installed manually on Selenium 4.
  • Reaching for Selenium when [[requests]] plus [[BeautifulSoup]] would do — it is far heavier.

Security Considerations

[!warning] Authorized use only Automate a browser only against applications you own or are explicitly authorized to test.

  • Selenium executes JavaScript from the target. You are running untrusted code in a real browser on your machine. Use headless mode, a disposable profile, and ideally a container or VM.
  • Never reuse your personal browser profile. Pointing Selenium at it exposes your real cookies, saved passwords, and history to the automated session.
  • Do not disable browser security features (--disable-web-security, --ignore-certificate-errors) outside a deliberately isolated lab.
  • Credentials typed with send_keys() may be captured in screenshots, page source dumps, and driver logs. Read them from the environment and avoid logging.
  • Browser automation is detectable and often prohibited by terms of service.
  • It is heavy. Each session is a full browser; running many in parallel will exhaust memory and can itself become a denial of service against the target.

Best Practices

  • Run headless (--headless=new) for automation and CI; add --no-sandbox in containers.
  • Always driver.quit() in a try/finally — orphaned browser processes leak memory.
  • Prefer explicit waits (WebDriverWait + expected_conditions) over time.sleep() for reliability.
  • Locate elements by stable attributes (id, name, data-*) rather than brittle XPaths.
  • Isolate sessions with a fresh user-data dir per run when testing auth flows.

References

Related Topics

  • [[requests]] — lighter option when JavaScript rendering isn't required
  • [[BeautifulSoup]] — parse driver.page_source after rendering
  • [[Readme|Python for Security Professionals]] — course home