Extract Json data from RestFul webservice using Python -

we use bugsplatsoftware.com collect crashes. have restfull web service returns json data. data individual crashes. data behind login/password...

i tried following results not expected.

import requests requests.auth import httpbasicauth  args={'id':11111,'data':0}  response=requests.get("https://www.bugsplatsoftware.com/individualcrash",params=args,auth=httpbasicauth("username","password"))  data=response.json() 

response.headers returns following

response.headers {'content-length': '10549', 'connection': 'close', 'server': 'apache/2.4.12 (win32) openssl/1.0.1l php/5.5.21', 'x-frame-options': 'sameorigin', 'x-pingback': 'https://www.bugsplatsoftware.com/xmlrpc.php', 'expires': 'thu, 19 nov 1981 08:52:00 gmt', 'cache-control': 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'x-xss-protection': '1; mode=block', 'date': 'wed, 22 apr 2015 17:43:37 gmt', 'content-encoding': 'gzip', 'link': '<https://www.bugsplatsoftware.com/?p=363>; rel=shortlink', 'vary': 'accept-encoding,user-agent', 'x-content-type-options': 'nosniff', 'x-powered-by': 'php/5.5.21', 'content-type': 'text/html; charset=utf-8', 'pragma': 'no-cache'} 

what need json data? in advance.

when print response.url shows https://www.bugsplatsoftware.com/login/ instead of https://www.bugsplatsoftware.com/individualcrash/?id=11111&data=0....

marmeladze, "bugsplatsoftware.com/individualcrash?id=11111&data=0"; returns json data (at least in browser) , need.

pygeek, when call response.content seems data html page.....

ivan, how specify "content-type" requests.get?

seems need using python requests: sessions, cookies, , post tried following

import requests s=requests.session() data={"login":'tester',"password":'testing'} url="https://wwww.bugsplatsoftware.com/login" r=s.post(url,data=data)  

and unauthorized error message

or if

s.get(url) many redirects

this pretty straight forward actually. json close in structure python lists , dictionaries. need convert json string receive web service call appropriate sequence type, can use list comprehension on extract want.

here sample code created call simple web service of mine.

import urllib, json, collections      def geturlasstring(url):     s = urllib.urlopen(url).read()     return s  def convertjsonstringtosequence(source):     j = json.jsondecoder(object_pairs_hook=collections.ordereddict).decode(source)     return j  def geturlasjsonsequence(url):     s = geturlasstring(url)     return convertjsonstringtosequence(s)  url = "http://127.0.0.1:8081/lookupnames" # local web service s = geturlasstring(url) print s j = geturlasjsonsequence(url) print j  samplejson = [{"lastname": "davinci", "firstname": "leonardo"}, {"lastname": "newton", "firstname": "isaac"}]  filteredlist = [e["firstname"] e in samplejson if e["lastname"].startswith("d")] print filteredlist 

much of functions created make easier, main parts this; need import json package. jsondecoder convert string web service call 1 of native python sequences (typically list of dictionaries). wrote bunch of helper functions either convert string sequence, or call url directly , return sequence.

the program wrote give following output:

[{"lastname": "davinci", "firstname": "leonardo"}] [ordereddict([(u'lastname', u'davinci'), (u'firstname', u'leonardo')])] ['leonardo']