pandas：按时间间隔平均滚动
我是pandas新手….我有一堆轮询数据; 我想计算一个滚动平均值来根据三天的窗口得到每一天的估计值。 正如我从这个问题所理解的，rolling_ *函数根据指定数量的值计算窗口，而不是特定的date时间范围。
是否有一个不同的函数来实现这个function？ 还是我坚持写我自己的？
编辑：
示例input数据：
polls_subset.tail(20) Out[185]: favorable unfavorable other enddate 2012-10-25 0.48 0.49 0.03 2012-10-25 0.51 0.48 0.02 2012-10-27 0.51 0.47 0.02 2012-10-26 0.56 0.40 0.04 2012-10-28 0.48 0.49 0.04 2012-10-28 0.46 0.46 0.09 2012-10-28 0.48 0.49 0.03 2012-10-28 0.49 0.48 0.03 2012-10-30 0.53 0.45 0.02 2012-11-01 0.49 0.49 0.03 2012-11-01 0.47 0.47 0.05 2012-11-01 0.51 0.45 0.04 2012-11-03 0.49 0.45 0.06 2012-11-04 0.53 0.39 0.00 2012-11-04 0.47 0.44 0.08 2012-11-04 0.49 0.48 0.03 2012-11-04 0.52 0.46 0.01 2012-11-04 0.50 0.47 0.03 2012-11-05 0.51 0.46 0.02 2012-11-07 0.51 0.41 0.00 输出每个date只有一行。
编辑x2：固定错字
那么这样的事情呢？
首先将dataframe重新采样为一维间隔。 这取决于所有重复date的值的平均值。 使用fill_method选项填写缺less的date值。 接下来，将重新采样的帧传递给pd.rolling_mean ，窗口为3，min_periods = 1：
pd.rolling_mean(df.resample("1D", fill_method="ffill"), window=3, min_periods=1) favorable unfavorable other enddate 2012-10-25 0.495000 0.485000 0.025000 2012-10-26 0.527500 0.442500 0.032500 2012-10-27 0.521667 0.451667 0.028333 2012-10-28 0.515833 0.450000 0.035833 2012-10-29 0.488333 0.476667 0.038333 2012-10-30 0.495000 0.470000 0.038333 2012-10-31 0.512500 0.460000 0.029167 2012-11-01 0.516667 0.456667 0.026667 2012-11-02 0.503333 0.463333 0.033333 2012-11-03 0.490000 0.463333 0.046667 2012-11-04 0.494000 0.456000 0.043333 2012-11-05 0.500667 0.452667 0.036667 2012-11-06 0.507333 0.456000 0.023333 2012-11-07 0.510000 0.443333 0.013333
更新 ：正如Ben在评论中指出的， pandas0.18.0的语法已经改变了 。 用新的语法，这将是：
df.resample("1d").sum().fillna(0).rolling(window=3, min_periods=1).mean()
我只是有同样的问题，但不规则间隔的数据点。 这里重新采样不是一个真正的select。 所以我创造了我自己的function。 也许这对其他人也有用：
from pandas import Series, DataFrame import pandas as pd from datetime import datetime, timedelta import numpy as np def rolling_mean(data, window, min_periods=1, center=False): ''' Function that computes a rolling mean Parameters ---------- data : DataFrame or Series If a DataFrame is passed, the rolling_mean is computed for all columns. window : int or string If int is passed, window is the number of observations used for calculating the statistic, as defined by the function pd.rolling_mean() If a string is passed, it must be a frequency string, eg '90S'. This is internally converted into a DateOffset object, representing the window size. min_periods : int Minimum number of observations in window required to have a value. Returns ------- Series or DataFrame, if more than one column ''' def f(x): '''Function to apply that actually computes the rolling mean''' if center == False: dslice = col[x-pd.datetools.to_offset(window).delta+timedelta(0,0,1):x] # adding a microsecond because when slicing with labels start and endpoint # are inclusive else: dslice = col[x-pd.datetools.to_offset(window).delta/2+timedelta(0,0,1): x+pd.datetools.to_offset(window).delta/2] if dslice.size < min_periods: return np.nan else: return dslice.mean() data = DataFrame(data.copy()) dfout = DataFrame() if isinstance(window, int): dfout = pd.rolling_mean(data, window, min_periods=min_periods, center=center) elif isinstance(window, basestring): idx = Series(data.index.to_pydatetime(), index=data.index) for colname, col in data.iterkv(): result = idx.apply(f) result.name = colname dfout = dfout.join(result, how='outer') if dfout.columns.size == 1: dfout = dfout.ix[:,0] return dfout # Example idx = [datetime(2011, 2, 7, 0, 0), datetime(2011, 2, 7, 0, 1), datetime(2011, 2, 7, 0, 1, 30), datetime(2011, 2, 7, 0, 2), datetime(2011, 2, 7, 0, 4), datetime(2011, 2, 7, 0, 5), datetime(2011, 2, 7, 0, 5, 10), datetime(2011, 2, 7, 0, 6), datetime(2011, 2, 7, 0, 8), datetime(2011, 2, 7, 0, 9)] idx = pd.Index(idx) vals = np.arange(len(idx)).astype(float) s = Series(vals, index=idx) rm = rolling_mean(s, window='2min')
user2689410的代码正是我所需要的。 提供我的版本（学分为user2689410），由于DataFrame中的整个行可以一次计算平均值，因此速度更快。
希望我的后缀约定是可读的：_s：string，_i：int，_b：bool，_ser：Series和_df：DataFrame。 在哪里可以find多个后缀，types可以是两个。
import pandas as pd from datetime import datetime, timedelta import numpy as np def time_offset_rolling_mean_df_ser(data_df_ser, window_i_s, min_periods_i=1, center_b=False): """ Function that computes a rolling mean Credit goes to user2689410 at http://stackoverflow.com/questions/15771472/pandas-rolling-mean-by-time-interval Parameters ---------- data_df_ser : DataFrame or Series If a DataFrame is passed, the time_offset_rolling_mean_df_ser is computed for all columns. window_i_s : int or string If int is passed, window_i_s is the number of observations used for calculating the statistic, as defined by the function pd.time_offset_rolling_mean_df_ser() If a string is passed, it must be a frequency string, eg '90S'. This is internally converted into a DateOffset object, representing the window_i_s size. min_periods_i : int Minimum number of observations in window_i_s required to have a value. Returns ------- Series or DataFrame, if more than one column >>> idx = [ ... datetime(2011, 2, 7, 0, 0), ... datetime(2011, 2, 7, 0, 1), ... datetime(2011, 2, 7, 0, 1, 30), ... datetime(2011, 2, 7, 0, 2), ... datetime(2011, 2, 7, 0, 4), ... datetime(2011, 2, 7, 0, 5), ... datetime(2011, 2, 7, 0, 5, 10), ... datetime(2011, 2, 7, 0, 6), ... datetime(2011, 2, 7, 0, 8), ... datetime(2011, 2, 7, 0, 9)] >>> idx = pd.Index(idx) >>> vals = np.arange(len(idx)).astype(float) >>> ser = pd.Series(vals, index=idx) >>> df = pd.DataFrame({'s1':ser, 's2':ser+1}) >>> time_offset_rolling_mean_df_ser(df, window_i_s='2min') s1 s2 2011-02-07 00:00:00 0.0 1.0 2011-02-07 00:01:00 0.5 1.5 2011-02-07 00:01:30 1.0 2.0 2011-02-07 00:02:00 2.0 3.0 2011-02-07 00:04:00 4.0 5.0 2011-02-07 00:05:00 4.5 5.5 2011-02-07 00:05:10 5.0 6.0 2011-02-07 00:06:00 6.0 7.0 2011-02-07 00:08:00 8.0 9.0 2011-02-07 00:09:00 8.5 9.5 """ def calculate_mean_at_ts(ts): """Function (closure) to apply that actually computes the rolling mean""" if center_b == False: dslice_df_ser = data_df_ser[ ts-pd.datetools.to_offset(window_i_s).delta+timedelta(0,0,1): ts ] # adding a microsecond because when slicing with labels start and endpoint # are inclusive else: dslice_df_ser = data_df_ser[ ts-pd.datetools.to_offset(window_i_s).delta/2+timedelta(0,0,1): ts+pd.datetools.to_offset(window_i_s).delta/2 ] if (isinstance(dslice_df_ser, pd.DataFrame) and dslice_df_ser.shape[0] < min_periods_i) or \ (isinstance(dslice_df_ser, pd.Series) and dslice_df_ser.size < min_periods_i): return dslice_df_ser.mean()*np.nan # keeps number format and whether Series or DataFrame else: return dslice_df_ser.mean() if isinstance(window_i_s, int): mean_df_ser = pd.rolling_mean(data_df_ser, window=window_i_s, min_periods=min_periods_i, center=center_b) elif isinstance(window_i_s, basestring): idx_ser = pd.Series(data_df_ser.index.to_pydatetime(), index=data_df_ser.index) mean_df_ser = idx_ser.apply(calculate_mean_at_ts) return mean_df_ser
这个例子似乎要求@andyhayden的评论中提出的加权平均值。 例如，10/25和10/27有两次民意测验。 如果你只是重新采样，然后采取平均值，那么10/26和10/27的投票权比10/25投票权的有效权重要高出两倍。
为了给每一个民意调查同样重要，而不是每天同样的重量，你可以做如下的事情。
>>> wt = df.resample('D',limit=5).count() favorable unfavorable other enddate 2012-10-25 2 2 2 2012-10-26 1 1 1 2012-10-27 1 1 1 >>> df2 = df.resample('D').mean() favorable unfavorable other enddate 2012-10-25 0.495 0.485 0.025 2012-10-26 0.560 0.400 0.040 2012-10-27 0.510 0.470 0.020
这给你做基于民意测验的意思，而不是一个基于日常的意思的原料。 和以前一样，民意调查是平均在10/25，但10/25的权重也被存储，是10/26或10/27的两倍，以反映10/25的两次民调。
>>> df3 = df2 * wt >>> df3 = df3.rolling(3,min_periods=1).sum() >>> wt3 = wt.rolling(3,min_periods=1).sum() >>> df3 = df3 / wt3 favorable unfavorable other enddate 2012-10-25 0.495000 0.485000 0.025000 2012-10-26 0.516667 0.456667 0.030000 2012-10-27 0.515000 0.460000 0.027500 2012-10-28 0.496667 0.465000 0.041667 2012-10-29 0.484000 0.478000 0.042000 2012-10-30 0.488000 0.474000 0.042000 2012-10-31 0.530000 0.450000 0.020000 2012-11-01 0.500000 0.465000 0.035000 2012-11-02 0.490000 0.470000 0.040000 2012-11-03 0.490000 0.465000 0.045000 2012-11-04 0.500000 0.448333 0.035000 2012-11-05 0.501429 0.450000 0.032857 2012-11-06 0.503333 0.450000 0.028333 2012-11-07 0.510000 0.435000 0.010000
请注意，10/27的滚动平均值现在是0.51500（轮询加权），而不是52.1667（白天加权）。
另请注意，从版本0.18.0开始，API已经对resample和rolling进行了更改。
滚动（pandas0.18.0新增function）
（pandas0.18.0新增function）
同时，增加了一个时间窗口能力。 请参阅下面的链接：
https://github.com/pydata/pandas/pull/13513
In [1]: df = DataFrame({'B': range(5)}) In [2]: df.index = [Timestamp('20130101 09:00:00'), ...: Timestamp('20130101 09:00:02'), ...: Timestamp('20130101 09:00:03'), ...: Timestamp('20130101 09:00:05'), ...: Timestamp('20130101 09:00:06')] In [3]: df Out[3]: B 2013-01-01 09:00:00 0 2013-01-01 09:00:02 1 2013-01-01 09:00:03 2 2013-01-01 09:00:05 3 2013-01-01 09:00:06 4 In [4]: df.rolling(2, min_periods=1).sum() Out[4]: B 2013-01-01 09:00:00 0.0 2013-01-01 09:00:02 1.0 2013-01-01 09:00:03 3.0 2013-01-01 09:00:05 5.0 2013-01-01 09:00:06 7.0 In [5]: df.rolling('2s', min_periods=1).sum() Out[5]: B 2013-01-01 09:00:00 0.0 2013-01-01 09:00:02 1.0 2013-01-01 09:00:03 3.0 2013-01-01 09:00:05 3.0 2013-01-01 09:00:06 7.0
我发现user2689410代码打破了，当我试图与window ='1M'作为商业月的三angular洲扔这个错误：
AttributeError: 'MonthEnd' object has no attribute 'delta'
我添加了直接传递相对时间增量的选项，所以您可以为用户定义的时间段做类似的事情。
感谢指针，这是我的尝试 – 希望它是有用的。
def rolling_mean(data, window, min_periods=1, center=False): """ Function that computes a rolling mean Reference: http://stackoverflow.com/questions/15771472/pandas-rolling-mean-by-time-interval Parameters ---------- data : DataFrame or Series If a DataFrame is passed, the rolling_mean is computed for all columns. window : int, string, Timedelta or Relativedelta int - number of observations used for calculating the statistic, as defined by the function pd.rolling_mean() string - must be a frequency string, eg '90S'. This is internally converted into a DateOffset object, and then Timedelta representing the window size. Timedelta / Relativedelta - Can directly pass a timedeltas. min_periods : int Minimum number of observations in window required to have a value. center : bool Point around which to 'center' the slicing. Returns ------- Series or DataFrame, if more than one column """ def f(x, time_increment): """Function to apply that actually computes the rolling mean :param x: :return: """ if not center: # adding a microsecond because when slicing with labels start # and endpoint are inclusive start_date = x - time_increment + timedelta(0, 0, 1) end_date = x else: start_date = x - time_increment/2 + timedelta(0, 0, 1) end_date = x + time_increment/2 # Select the date index from the dslice = col[start_date:end_date] if dslice.size < min_periods: return np.nan else: return dslice.mean() data = DataFrame(data.copy()) dfout = DataFrame() if isinstance(window, int): dfout = pd.rolling_mean(data, window, min_periods=min_periods, center=center) elif isinstance(window, basestring): time_delta = pd.datetools.to_offset(window).delta idx = Series(data.index.to_pydatetime(), index=data.index) for colname, col in data.iteritems(): result = idx.apply(lambda x: f(x, time_delta)) result.name = colname dfout = dfout.join(result, how='outer') elif isinstance(window, (timedelta, relativedelta)): time_delta = window idx = Series(data.index.to_pydatetime(), index=data.index) for colname, col in data.iteritems(): result = idx.apply(lambda x: f(x, time_delta)) result.name = colname dfout = dfout.join(result, how='outer') if dfout.columns.size == 1: dfout = dfout.ix[:, 0] return dfout
并用3天的时间窗口来计算均值的例子：
from pandas import Series, DataFrame import pandas as pd from datetime import datetime, timedelta import numpy as np from dateutil.relativedelta import relativedelta idx = [datetime(2011, 2, 7, 0, 0), datetime(2011, 2, 7, 0, 1), datetime(2011, 2, 8, 0, 1, 30), datetime(2011, 2, 9, 0, 2), datetime(2011, 2, 10, 0, 4), datetime(2011, 2, 11, 0, 5), datetime(2011, 2, 12, 0, 5, 10), datetime(2011, 2, 12, 0, 6), datetime(2011, 2, 13, 0, 8), datetime(2011, 2, 14, 0, 9)] idx = pd.Index(idx) vals = np.arange(len(idx)).astype(float) s = Series(vals, index=idx) # Now try by passing the 3 days as a relative time delta directly. rm = rolling_mean(s, window=relativedelta(days=3)) >>> rm Out[2]: 2011-02-07 00:00:00 0.0 2011-02-07 00:01:00 0.5 2011-02-08 00:01:30 1.0 2011-02-09 00:02:00 1.5 2011-02-10 00:04:00 3.0 2011-02-11 00:05:00 4.0 2011-02-12 00:05:10 5.0 2011-02-12 00:06:00 5.5 2011-02-13 00:08:00 6.5 2011-02-14 00:09:00 7.5 Name: 0, dtype: float64
为了保持它的基本，我使用了一个循环和类似的东西，让你开始（我的索引是date时间）：
import pandas as pd import datetime as dt #populate your dataframe: "df" #... df[df.index<(df.index[0]+dt.timedelta(hours=1))] #gives you a slice. you can then take .sum() .mean(), whatever
然后你可以在这个片上运行函数。 你可以看到如何添加一个迭代器来创build窗口的开始，而不是数据框索引中的第一个值，然后滚动窗口（例如，你也可以使用一个>开始的规则）。
请注意，对于超大型数据或非常小的增量，这可能效率较低，因为您的切片可能会变得更加费力（对于我来说，对于数十万行数据和几个列，尽pipe对于几个星期的小时窗来说足够好）
