用特定模式从txt文件创建熊猫数据框
我需要基于以下结构创建基于文本文件的Pandas DataFrame：
Alabama[edit] Auburn (Auburn University)[1] Florence (University of North Alabama) Jacksonville (Jacksonville State University)[2] Livingston (University of West Alabama)[2] Montevallo (University of Montevallo)[2] Troy (Troy University)[2] Tuscaloosa (University of Alabama, Stillman College, Shelton State)[3][4] Tuskegee (Tuskegee University)[5] Alaska[edit] Fairbanks (University of Alaska Fairbanks)[2] Arizona[edit] Flagstaff (Northern Arizona University)[6] Tempe (Arizona State University) Tucson (University of Arizona) Arkansas[edit] 具有“[edit]”的行是国家，行[number]是地区。 我需要拆分以下内容，然后重复每个区域名称的州名称。
Index State Region Name 0 Alabama Aurburn... 1 Alabama Florence... 2 Alabama Jacksonville... ... 9 Alaska Fairbanks... 10 Alaska Arizona... 11 Alaska Flagstaff...
熊猫DataFrame
我不知道如何根据“[编辑]”和“[编号]”或“（字符）”将文本文件拆分到相应的列中，并重复每个区域名称的状态名称。 请任何人都可以给我一个起点，开始完成以下内容。
您可以先使用参数name DataFrame来创建带有列Region Name DataFrame ，分隔符是DataFrame值中的值（如; ）：
df = pd.read_csv('filename.txt', sep=";", names=['Region Name'])
然后insert新的列State与extract行的文本[edit]和所有的值(从最后到列Region Name 。
df.insert(0, 'State', df['Region Name'].str.extract('(.*)\[edit\]', expand=False).ffill()) df['Region Name'] = df['Region Name'].str.replace(r' \(.+$', '')
最后删除行boolean indexing文本[edit] ，由str.contains创建str.contains ：
df = df[~df['Region Name'].str.contains('\[edit\]')].reset_index(drop=True) print (df) State Region Name 0 Alabama Auburn 1 Alabama Florence 2 Alabama Jacksonville 3 Alabama Livingston 4 Alabama Montevallo 5 Alabama Troy 6 Alabama Tuscaloosa 7 Alabama Tuskegee 8 Alaska Fairbanks 9 Arizona Flagstaff 10 Arizona Tempe 11 Arizona Tucson
如果需要所有值解决方案更容易：
df = pd.read_csv('filename.txt', sep=";", names=['Region Name']) df.insert(0, 'State', df['Region Name'].str.extract('(.*)\[edit\]', expand=False).ffill()) df = df[~df['Region Name'].str.contains('\[edit\]')].reset_index(drop=True) print (df) State Region Name 0 Alabama Auburn (Auburn University)[1] 1 Alabama Florence (University of North Alabama) 2 Alabama Jacksonville (Jacksonville State University)[2] 3 Alabama Livingston (University of West Alabama)[2] 4 Alabama Montevallo (University of Montevallo)[2] 5 Alabama Troy (Troy University)[2] 6 Alabama Tuscaloosa (University of Alabama, Stillman Co... 7 Alabama Tuskegee (Tuskegee University)[5] 8 Alaska Fairbanks (University of Alaska Fairbanks)[2] 9 Arizona Flagstaff (Northern Arizona University)[6] 10 Arizona Tempe (Arizona State University) 11 Arizona Tucson (University of Arizona)
假设你有以下DF：
In [73]: df Out[73]: text 0 Alabama[edit] 1 Auburn (Auburn University)[1] 2 Florence (University of North Alabama) 3 Jacksonville (Jacksonville State University)[2] 4 Livingston (University of West Alabama)[2] 5 Montevallo (University of Montevallo)[2] 6 Troy (Troy University)[2] 7 Tuscaloosa (University of Alabama, Stillman Co... 8 Tuskegee (Tuskegee University)[5] 9 Alaska[edit] 10 Fairbanks (University of Alaska Fairbanks)[2] 11 Arizona[edit] 12 Flagstaff (Northern Arizona University)[6] 13 Tempe (Arizona State University) 14 Tucson (University of Arizona) 15 Arkansas[edit]
你可以使用Series.str.extract（）方法：
In [117]: df['State'] = df.loc[df.text.str.contains('[edit]', regex=False), 'text'].str.extract(r'(.*?)\[edit\]', expand=False) In [118]: df['Region Name'] = df.loc[df.State.isnull(), 'text'].str.extract(r'(.*?)\s*[\(\[]+.*[\n]*', expand=False) In [120]: df.State = df.State.ffill() In [121]: df Out[121]: text State Region Name 0 Alabama[edit] Alabama NaN 1 Auburn (Auburn University)[1] Alabama Auburn 2 Florence (University of North Alabama) Alabama Florence 3 Jacksonville (Jacksonville State University)[2] Alabama Jacksonville 4 Livingston (University of West Alabama)[2] Alabama Livingston 5 Montevallo (University of Montevallo)[2] Alabama Montevallo 6 Troy (Troy University)[2] Alabama Troy 7 Tuscaloosa (University of Alabama, Stillman Co... Alabama Tuscaloosa 8 Tuskegee (Tuskegee University)[5] Alabama Tuskegee 9 Alaska[edit] Alaska NaN 10 Fairbanks (University of Alaska Fairbanks)[2] Alaska Fairbanks 11 Arizona[edit] Arizona NaN 12 Flagstaff (Northern Arizona University)[6] Arizona Flagstaff 13 Tempe (Arizona State University) Arizona Tempe 14 Tucson (University of Arizona) Arizona Tucson 15 Arkansas[edit] Arkansas NaN In [122]: df = df.dropna() In [123]: df Out[123]: text State Region Name 1 Auburn (Auburn University)[1] Alabama Auburn 2 Florence (University of North Alabama) Alabama Florence 3 Jacksonville (Jacksonville State University)[2] Alabama Jacksonville 4 Livingston (University of West Alabama)[2] Alabama Livingston 5 Montevallo (University of Montevallo)[2] Alabama Montevallo 6 Troy (Troy University)[2] Alabama Troy 7 Tuscaloosa (University of Alabama, Stillman Co... Alabama Tuscaloosa 8 Tuskegee (Tuskegee University)[5] Alabama Tuskegee 10 Fairbanks (University of Alaska Fairbanks)[2] Alaska Fairbanks 12 Flagstaff (Northern Arizona University)[6] Arizona Flagstaff 13 Tempe (Arizona State University) Arizona Tempe 14 Tucson (University of Arizona) Arizona Tucson
您可以先将文件解析为元组：
import pandas as pd from collections import namedtuple Item = namedtuple('Item', 'state area') items = [] with open('unis.txt') as f: for line in f: l = line.rstrip('\n') if l.endswith('[edit]'): state = l.rstrip('[edit]') else: i = l.index(' (') area = l[:i] items.append(Item(state, area)) df = pd.DataFrame.from_records(items, columns=['State', 'Area']) print df
输出：
State Area 0 Alabama Auburn 1 Alabama Florence 2 Alabama Jacksonville 3 Alabama Livingston 4 Alabama Montevallo 5 Alabama Troy 6 Alabama Tuscaloosa 7 Alabama Tuskegee 8 Alaska Fairbanks 9 Arizona Flagstaff 10 Arizona Tempe 11 Arizona Tucson
TL; DR
s.groupby(s.str.extract('(?P<State>.*?)\[edit\]', expand=False).ffill()).apply(pd.Series.tail, n=-1).reset_index(name='Region_Name').iloc[:, [0, 2]]
regex = '(?P<State>.*?)\[edit\]' # pattern to match print(s.groupby( # will get nulls where we don't have "[edit]" # forward fill fills in the most recent line # where we did have an "[edit]" s.str.extract(regex, expand=False).ffill() ).apply( # I still have all the original values # If I group by the forward filled rows # I'll want to drop the first one within each group pd.Series.tail, n=-1 ).reset_index( # munge the dataframe to get columns sorted name='Region_Name' )[['State', 'Region_Name']]) State Region_Name 0 Alabama Auburn (Auburn University)[1] 1 Alabama Florence (University of North Alabama) 2 Alabama Jacksonville (Jacksonville State University)[2] 3 Alabama Livingston (University of West Alabama)[2] 4 Alabama Montevallo (University of Montevallo)[2] 5 Alabama Troy (Troy University)[2] 6 Alabama Tuscaloosa (University of Alabama, Stillman Co... 7 Alabama Tuskegee (Tuskegee University)[5] 8 Alaska Fairbanks (University of Alaska Fairbanks)[2] 9 Arizona Flagstaff (Northern Arizona University)[6] 10 Arizona Tempe (Arizona State University) 11 Arizona Tucson (University of Arizona)
建立
txt = """Alabama[edit] Auburn (Auburn University)[1] Florence (University of North Alabama) Jacksonville (Jacksonville State University)[2] Livingston (University of West Alabama)[2] Montevallo (University of Montevallo)[2] Troy (Troy University)[2] Tuscaloosa (University of Alabama, Stillman College, Shelton State)[3][4] Tuskegee (Tuskegee University)[5] Alaska[edit] Fairbanks (University of Alaska Fairbanks)[2] Arizona[edit] Flagstaff (Northern Arizona University)[6] Tempe (Arizona State University) Tucson (University of Arizona) Arkansas[edit]""" s = pd.read_csv(StringIO(txt), sep='|', header=None, squeeze=True)
在将文件放入数据框之前，您可能需要对文件执行一些额外的操作。
一个起点是将文件拆分成行，在每行中搜索字符串[edit] ，将字符串名称作为字典的关键字。
我不认为熊猫有任何内置的方法来处理这种格式的文件。
