Skip to main content

All Questions

Tagged with
286 votes
17 answers
485k views

How can I scrape a page with dynamic content (created by JavaScript) in Python?

I'm trying to develop a simple web scraper. I want to extract plain text without HTML markup. My code works on plain (static) HTML, but not when content is generated by JavaScript embedded in the page....
mocopera's user avatar
  • 2,913
635 votes
26 answers
484k views

Can a website detect when you are using Selenium with chromedriver?

I've been testing out Selenium with Chromedriver and I noticed that some pages can detect that you're using Selenium even though there's no automation at all. Even when I'm just browsing manually just ...
Ryan Weinstein's user avatar
313 votes
24 answers
132k views

Javascript equivalent of Python's zip function

Is there a javascript equivalent of Python's zip function? That is, given multiple arrays of equal lengths create an array of pairs. For instance, if I have three arrays that look like this: var ...
pq.'s user avatar
  • 3,485
106 votes
13 answers
248k views

Python: FastAPI error 422 with POST request when sending JSON data

I'm building a simple API to test a database. When I use GET request everything works fine, but if I change to POST, I get 422 Unprocessable Entity error. Here is the FastAPI code: from fastapi import ...
Smith's user avatar
  • 1,328
10 votes
3 answers
9k views

JavaScript raises SyntaxError with data rendered in Jinja template

I am trying to pass data as JSON from a Flask route to a Jinja template rendering JavaScript. I want to iterate over the data using JavaScript. The browser shows SyntaxError: Unexpected token '&'...
EspenG's user avatar
  • 537
168 votes
10 answers
151k views

Can scrapy be used to scrape dynamic content from websites that are using AJAX?

I have recently been learning Python and am dipping my hand into building a web-scraper. It's nothing fancy at all; its only purpose is to get the data off of a betting website and have this data put ...
Joseph's user avatar
  • 3,949
315 votes
19 answers
371k views

How can I use the variables from "views.py" in JavasScript, "<script></script>" in a Django template?

When I render a page using the Django template renderer, I can pass in a dictionary variable containing various values to manipulate them in the page using {{ myVar }}. Is there a way to access the ...
AlMcLean's user avatar
  • 3,509
6 votes
1 answer
7k views

FastAPI is not returning cookies to React frontend

Why doesn't FastAPI return the cookie to my frontend, which is a React app? Here is my code: @router.post("/login") def user_login(response: Response,username :str = Form(),password :str = ...
jabodom934's user avatar
28 votes
3 answers
31k views

Display data streamed from a Flask view as it updates

I have a view that generates data and streams it in real time. I can't figure out how to send this data to a variable that I can use in my HTML template. My current solution just outputs the data to ...
JeffThompson's user avatar
  • 1,590
160 votes
5 answers
73k views

WebDriver click() vs JavaScript click()

The Story: Here on StackOverflow, I've seen users reporting that they cannot click an element via selenium WebDriver "click" command and can work around it with a JavaScript click by executing a ...
alecxe's user avatar
  • 471k
160 votes
9 answers
267k views

How can I pass data from Flask to JavaScript in a template?

My app makes a call to an API that returns a dictionary. I want to pass information from this dict to JavaScript in the view. I am using the Google Maps API in the JS, specifically, so I'd like to ...
mea's user avatar
  • 1,603
112 votes
3 answers
198k views

Running javascript in Selenium using Python

I am totally new to Selenium. I want to execute a javascript snippet in the following code(as commented in the code), but can't do so. Please help. from selenium import webdriver import selenium from ...
theharshest's user avatar
  • 7,837
413 votes
13 answers
223k views

JSON datetime between Python and JavaScript

I want to send a datetime.datetime object in serialized form from Python using JSON and de-serialize in JavaScript using JSON. What is the best way to do this?
user avatar
121 votes
7 answers
377k views

Call Python function from JavaScript code

I'd like to call a Python function from JavaScript code, because there isn't an alternative in JavaScript for doing what I want. Is this possible? Could you adjust the below snippet to work? ...
xralf's user avatar
  • 3,850
18 votes
5 answers
18k views

scrape html generated by javascript with python

I need to scrape a site with python. I obtain the source html code with the urlib module, but I need to scrape also some html code that is generated by a javascript function (which is included in the ...
hymloth's user avatar
  • 6,997

15 30 50 per page
1
2 3 4 5
98