웹사이트의 html 코드 가져오는 방법 2가지
1. with 키워드 사용하기
웹사이트에서 오른쪽 마우스 클릭> 다른 이름으로 저장>html문서로 저장하기
작업하는 파일과 같은 폴더에 저장하기
with키워드로 파일 불러오기
에러가 발생하면 인코딩 UTF-8을 써준다.
with open("The 100 Greatest Movies.html", encoding="UTF-8") as f:
response= f.read()
2. requests 패키지 사용하기
- requests 패키지 import하기
import requests
- URL 가져오기
URL = "https://web.archive.org/web/20200518073855/https://www.empireonline.com/movies/features/best-movies-2/"
- get 메소드로 데이터 가져오기
response2 = requests.get(URL)
- text 파일로 가져오기
website_html2 = response2.text
웹스크래핑하기
soup = BeautifulSoup(website_html2, "html.parser")
all_data = soup.find_all(name ="h3", class_="title")
movie_top100 = []
for data in all_data:
movie_top100.append(data.getText())
movie_top100.reverse()
print(movie_top100)>>['1) The Godfather', '2) The Empire Strikes Back', '3) The Dark Knight', '4) The Shawshank Redemption', '5) Pulp Fiction', '6) Goodfellas', '7) Raiders Of The Lost Ark', '8) Jaws', '9) Star Wars', '10) The Lord Of The Rings: The Fellowship Of The Ring', '11) Back To The Future', '12: The Godfather Part II', '13) Blade Runner', '14) Alien', '15) Aliens', '16) The Lord Of The Rings: The Return Of The King', '17) Fight Club', '18) Inception', '19) Jurassic Park', '20) Die Hard', '21) 2001: A Space Odyssey', '22) Apocalypse Now', '23) The Lord Of The Rings: The Two Towers', '24) The Matrix', '25) Terminator 2: Judgment Day', '26) Heat', '27) The Good, The Bad And The Ugly', '28) Casablanca', '29) The Big Lebowski', '30) Seven', '31) Taxi Driver', '32) The Usual Suspects', "33) Schindler's List", '34) Guardians Of The Galaxy', '35) The Shining', '36) The Departed', '37) The Thing', '38) Mad Max: Fury Road', '39) Saving Private Ryan', '40) 12 Angry Men', '41) Eternal Sunshine Of The Spotless Mind', '42) There Will Be Blood', "43) One Flew Over The Cuckoo's Nest", '44) Gladiator', '45) Drive', '46) Citizen Kane', '47) Interstellar', '48) The Silence Of The Lambs', '49) Trainspotting', '50) Lawrence Of Arabia', "51) It's A Wonderful Life", '52) Once Upon A Time In The West', '53) Psycho', '54) Vertigo', "55) Pan's Labyrinth", '56) Reservoir Dogs', '57) Whiplash', '58) Inglourious Basterds', '59) E.T. â\x80\x93 The Extra Terrestrial', '60) American Beauty', '61) Forrest Gump', '62) La La Land', '63) Donnie Darko', '64) L.A. Confidential', '65) Avengers Assemble', '66) Return Of The Jedi', '67) Memento', '68) Ghostbusters', "69) Singin' In The Rain", '70) The Lion King', '71) Hot Fuzz', '72) Rear Window', '73) Seven Samurai', '74) Mulholland Dr.', '75) Fargo', '76) A Clockwork Orange', '77) Toy Story', '78) Oldboy', '79) Captain America: Civil War', '15) Spirited Away', '81) The Social Network', '82) Some Like It Hot', '83) True Romance', '84) Rocky', '85) Léon', '86) Indiana Jones And The Last Crusade', '87) Predator', '88) The Exorcist', '89) Shaun Of The Dead', '90) No Country For Old Men', '91) The Prestige', '92) The Terminator', '93) The Princess Bride', '94) Lost In Translation', '95) Arrival', '96) Good Will Hunting', '97) Titanic', '98) Amelie', '99) Raging Bull', '100) Stand By Me']
새로운 text 파일 만들기
with open("Best Movie Top100.txt", mode ="w", encoding="UTF-8") as f:
for li in movie_top100:
f.write(f"{li}\n")