pandas

JFLゴール数ランキング(GoogleDrive)

imabari.hateblo.jp import csv import time from urllib.parse import urljoin import gspread import pandas as pd import requests from bs4 import BeautifulSoup from oauth2client.service_account import ServiceAccountCredentials from tqdm import…

JFLランキング作成(GoogleDrive)

pip3 install pandas[ pip3 install beautifulsoup4 pip3 install html5lib pip3 install lxml pip3 install black pip3 install gspread pip3 install oauth2clinet pip3 install tqdm # -*- coding: utf-8 -*- import gspread import pandas as pd import …

Pandasで愛媛の河川の水位を取得

github.com import datetime import time import pandas as pd from tqdm import tqdm # 時間 def timeconv(x): H, M = map(int, x.split(":")) return datetime.timedelta(hours=H, minutes=M) # 設定 # 東予東部 grp, tpg = "USR019", 2 # 東予西部 grp, t…

Pandasで愛媛のダム情報を取得

github.com import datetime import time import pandas as pd from tqdm import tqdm # 時間 def timeconv(x): H, M = map(int, x.split(":")) return datetime.timedelta(hours=H, minutes=M) # GRP = USR004:玉川ダム、USR005:台ダム、USR010:鹿野川ダム…

ボランティア数集計

!pip install lxml !pip install seaborn !apt install fonts-ipafont-gothic !rm /content/.cache/matplotlib/fontList.json import pandas as pd url = 'https://ehimesvc.jp/?p=70' dfs = pd.read_html(url, index_col=0, na_values=['活動中止', '終了',…

JFLの試合結果分析

github.com

FC今治のゴール集計・先取点

!pip install lxml !apt install fonts-ipafont-gothic !rm /content/.cache/matplotlib/fontList.json """再起動""" import time import csv import requests from bs4 import BeautifulSoup # 試合数 n = 10 + 1 with open('fcimabari_goal.tsv', 'w') as …

PythonでスクレイピングしてテーブルをCSVに保存

Beautifulsoupの場合 import csv from bs4 import BeautifulSoup import requests url = 'http://www.example.com/' r = requests.get(url) if r.status_code == requests.codes.ok: soup = BeautifulSoup(r.content, 'html.parser') result = [[[td.get_tex…

JFLの試合結果からランキングを作成

import pandas as pd url = 'http://www.jfl.or.jp/jfl-pc/view/s.php?a=1411&f=2019A001_spc.html' dfs = pd.read_html(url, skiprows=1, na_values='-') len(dfs) df = pd.concat( dfs, keys=[i for i in range(1, len(dfs) + 1)], names=['節', '番号']) …

Pandasでミニロトで遊ぶ

!pip install lxml import pandas as pd import io import requests # User-agentを設定しないとダウンロードできない url = 'http://www.japannetbank.co.jp/lottery/co/minilotojnb.csv' headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64;…

ミニロトの結果を取得

TSV保存 from selenium import webdriver from selenium.webdriver.firefox.options import Options from bs4 import BeautifulSoup import csv options = Options() options.set_headless() driver = webdriver.Firefox(options=options) driver.get( 'http…

pandas

qiita.com news.mynavi.jp amalog.hateblo.jp pandas: powerful Python data analysis toolkit — pandas 0.23.0 documentation www.atmarkit.co.jp www.atmarkit.co.jp www.atmarkit.co.jp # 型確認 df.dtypes # 欠損値のカウント df.isnull().sum() # 欠損…

石手川ダムのリアルタイムダム諸量一覧表でpandas練習

!pip install lxml import pandas as pd import requests from urllib.parse import urljoin from bs4 import BeautifulSoup # 石手川ダムのリアルタイムダム諸量一覧表 url = 'http://www1.river.go.jp/cgi-bin/DspDamData.exe?ID=1368080150020&KIND=3&PAG…

FC今治の順位をスクレイピング

import datetime import requests from bs4 import BeautifulSoup url = 'http://www.jfl.or.jp/jfl-pc/view/s.php?a=1277' r = requests.get(url) if r.status_code == requests.codes.ok: soup = BeautifulSoup(r.content, 'html.parser') update_str = so…

Colaboratoryでスクレイピング

news.mynavi.jp https://colab.research.google.com/ import pandas as pd import requests from bs4 import BeautifulSoup url = 'http://www.river.go.jp/kawabou/ipDamGaikyo.do?init=init&areaCd=88&prefCd=3801&townCd=&gamenId=01-0903&fldCtlParty=no…