Python software development (CSV to Pandas to SQL or CSV to SQL to Pandas) -

-

i have multiple csv files want manipulate(calculate mean, sum, etc.) , after want store them in sqlite database.

but want know proper way that.

csv sql , manipulate pandas

or

csv, manipulate pandas , store in sql

for example, want store data in table http://financials.morningstar.com/ratios/r.html?t=goog&region=usa&culture=en-us. update yearly , add 2013,2014,etc. in sql table. i'll create column have 10 year average each rows, etc.

regards,

right now, pandas's support manipulating csv files far beyond of ability manipulate sql databases, though there significant effort right improve latter leaps , bounds!

take @ the read_csv docs. 1 of flexible, fast, , powerful text file readers/writers out there data analysis applications.

on other hand read_sql can write sqlite databases , doesn't store index.

with read_csv can read multiindex objects (hierarchical indexes)!

if aren't tied database, recommend using pandas hdf5 persistence. use time , works great.

in summary, recommend first choice listed if must use database since writing database not going work unless you're using sqlite, can read others besides sqlite.


Comments

Post a Comment

Popular posts from this blog

plot - Remove Objects from Legend When You Have Also Used Fit, Matlab -

-
Image
i've plotted data points , fitted exponential curve between them using 'fit' in matlab. problem fit-function got fitted line wanted plot, markers on top of regular markers. solved problem plotting wanted markers on top of unwanted couldn't seen. problem. when want show legends dots in there. how can remove markers legend without removing fitted line since both of them hidden inside fit-function? can stop 'fit' plotting unwanted markers? so, want remove blue dot called 'hoff' in picture below. you can leave out legend-entries manually leaving out handles of lines, dont want in legend. try this: %if plot something, want showing in legend, save handle: h_line = plot(x,y) %you dont want show line? dont anything, plot it: plot(mymarker) %then set legend-> can add text legend-entries %a cell-array containing strings want show up: legend([h_line another_line],{'text1' 'text2'}) with example (see comments) came solution: ...
Read more

java - Why does my date parsing return a weird date? -

-
here code using: string string = "08/07/2013".replace('/', '-'); date date = new simpledateformat("yyyy-mm-dd").parse(string); why date return: "wen jan 3 00:00:00 est 14"? not @ date format told use. edit: need format because database using requires format. the format use parse date string, not match it. using yyyy 08 . use following format: new simpledateformat("dd-mm-yyyy") and why @ replacing / - ? can build pattern original string only: string string = "08/07/2013" date date = new simpledateformat("dd/mm/yyyy").parse(string); and if want date string in yyyy-mm-dd format, can format date using dateformat#format(date) method: string formatteddate = new simpledateformat("yyyy-mm-dd").format(date); see also: change date format in java string simpledateformat java doc
Read more

Need help in packaging app using TideSDK on Windows -

-
i developed application using tidesdk on windows , when click launchapp works fine. moved package , installed imagemagick , wix 3.0 required packaging on windows. clicked on package runtime , cannot guess packaged app is? found file 'installer' in workspace when click on it says installer not find application path . think application not being packaged properly. here log see in tidesdk: preparing package desktop app runtime. 1 moment... staging trial -> copying contents d:\lab\tidesdk\trial d:\lab\tidesdk\trial\packages\win32\bundle\trial -> copying installer c:\programdata\tidesdk\sdk\win32\1.3.1-beta\installer d:\lab\tidesdk\trial\packages\win32\bundle\trial -> copying runtime d:\lab\tidesdk\trial\packages\win32\bundle\trial -> date: d:\lab\tidesdk\trial\packages\win32\bundle\trial\modules\app\1.3.1-beta -> date: d:\lab\tidesdk\trial\packages\win32\bundle\trial\modules\codec\1.3.1-beta -> date: d:\lab\tidesdk\trial\packages\win32\bundle\trial\mod...
Read more