pandas

JFLの試合結果分析

github.com

FC今治のゴール集計・先取点

!pip install lxml !apt install fonts-ipafont-gothic !rm /content/.cache/matplotlib/fontList.json """再起動""" import time import csv import requests from bs4 import BeautifulSoup # 試合数 n = 10 + 1 with open('fcimabari_goal.tsv', 'w') as …

PythonでスクレイピングしてテーブルをCSVに保存

Beautifulsoupの場合 import csv from bs4 import BeautifulSoup import requests url = 'http://www.example.com/' r = requests.get(url) if r.status_code == requests.codes.ok: soup = BeautifulSoup(r.content, 'html.parser') result = [[[td.get_tex…

JFLの試合結果からランキングを作成

import numpy as np import pandas as pd url = 'http://www.jfl.or.jp/jfl-pc/view/s.php?a=1270&f=2018A001_spc.html' dfs = pd.read_html(url, skiprows=1, na_values='-') len(dfs) df = pd.concat( dfs, keys=[i for i in range(1, len(dfs) + 1)], nam…

Pandasでミニロトで遊ぶ

!pip install lxml import pandas as pd import io import requests # User-agentを設定しないとダウンロードできない url = 'http://www.japannetbank.co.jp/lottery/co/minilotojnb.csv' headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64;…

ミニロトの結果を取得

TSV保存 from selenium import webdriver from selenium.webdriver.firefox.options import Options from bs4 import BeautifulSoup import csv options = Options() options.set_headless() driver = webdriver.Firefox(options=options) driver.get( 'http…

pandas

qiita.com news.mynavi.jp amalog.hateblo.jp pandas: powerful Python data analysis toolkit — pandas 0.23.0 documentation www.atmarkit.co.jp www.atmarkit.co.jp www.atmarkit.co.jp # 型確認 df.dtypes # 欠損値のカウント df.isnull().sum() # 欠損…

石手川ダムのリアルタイムダム諸量一覧表でpandas練習

!pip install lxml import pandas as pd import requests from urllib.parse import urljoin from bs4 import BeautifulSoup # 石手川ダムのリアルタイムダム諸量一覧表 url = 'http://www1.river.go.jp/cgi-bin/DspDamData.exe?ID=1368080150020&KIND=3&PAG…

FC今治の順位をスクレイピング

import datetime import requests from bs4 import BeautifulSoup url = 'http://www.jfl.or.jp/jfl-pc/view/s.php?a=1277' r = requests.get(url) if r.status_code == requests.codes.ok: soup = BeautifulSoup(r.content, 'html.parser') update_str = so…

Colaboratoryでスクレイピング

news.mynavi.jp https://colab.research.google.com/ import pandas as pd import requests from bs4 import BeautifulSoup url = 'http://www.river.go.jp/kawabou/ipDamGaikyo.do?init=init&areaCd=88&prefCd=3801&townCd=&gamenId=01-0903&fldCtlParty=no…