User Controls
Py fags, I need help...
-
2017-02-25 at 4:46 AM UTCSo am trying to vote on a poll, but it doesn't seem to be regisring the vote, I think I need to ad a payload or some shit, here's my example
status_id = '835306569516507138'
url = 'https://twitter.com/StarsPoll/status/'+status_id
code = bs(session.get(url).text, 'html.parser')
auth_token = code.find("input", {"name": "authenticity_token"}).attrs['value']
#card_name = code.find("div", {"data-card2-type"}).value
card_name = 'poll2choice_text_only'
data = {
'tweet_id': status_id,
'card_name': card_name,
'authenticity_token': auth_token,
'forward': 'true',
'capi_uri': 'capi%3A%2F%2Fpassthrough%2F1'
}
payload = {
}
#this is payload request when on network tab in chrome when voting?
#"twitter:string:card_uri":"card://835331856526684160","twitter:long:original_tweet_id":"835331860725186560","twitter:string:selected_choice":"2"
headers = {
"Host": "www.twitter.com",
"Origin": "https://www.twitter.com",
"X-Requested-With": "XMLHttpRequest",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5"
}
voteurl = 'https://twitter.com/i/cards/api/v1.json?tweet_id='+status_id+'&card_name='+card_name+'&authenticity_token='+auth_token+'&forward=true&capi_uri=capi%3A%2F%2Fpassthrough%2F1'
vote = session.post(voteurl, data = data, headers=headers, cookies=session.cookies)
I'm getting a 200 status code, but still no vote.
-
2017-02-25 at 4:48 AM UTCdid u try turning it off and back on again?
-
2017-02-25 at 5 AM UTCI'm not familiar with the library you're using. Is the vote button JS based? Or HTML? If it's HTML you should just use the Mechanize lib imo.
-
2017-02-25 at 5:01 AM UTCWhere did you get that url? I think the v1 api has been deprecated for a while and I can't find any documentation it.
-
2017-02-25 at 5:02 AM UTCOh wait, it's just the endpoint your browser is using?
What's the response body? -
2017-02-25 at 2:46 PM UTC
Originally posted by Lanny Oh wait, it's just the endpoint your browser is using?
What's the response body?
the source code:
view-source:https://twitter.com/StarsPoll/status/835306569516507138 -
2017-02-25 at 3:14 PM UTC
-
2017-02-25 at 8:39 PM UTCOh, I see the issue, although you didn't actually post your source...
Take a look at the content-type header on the request you make through chrome vs. the one your script is producing. In the browser it's probably going to be text/plain (which is used to transmit a JSON string) while passing a dictionary to requests will probably send it as application/x-www-form-urlencoded. Not sure how smart requests is, you might be able to set the header yourself and json.dumps your data or maybe there's more ceremony involved but I'm sure it's in the docs somewhere. The important thing to note is that while the response's content-type is application/json your request is not, I'm not sure why twitter did this, it doesn't really matter as far as I can tell, but you probably will need to follow their scheme and send as text/plain that just happens to be valid JSON.
FYI this has nothing to do with python -
2017-02-25 at 8:52 PM UTCWhat a fucking nood!
-
2017-02-25 at 9:38 PM UTC
Originally posted by Lanny Oh, I see the issue, although you didn't actually post your source…
Take a look at the content-type header on the request you make through chrome vs. the one your script is producing. In the browser it's probably going to be text/plain (which is used to transmit a JSON string) while passing a dictionary to requests will probably send it as application/x-www-form-urlencoded. Not sure how smart requests is, you might be able to set the header yourself and json.dumps your data or maybe there's more ceremony involved but I'm sure it's in the docs somewhere. The important thing to note is that while the response's content-type is application/json your request is not, I'm not sure why twitter did this, it doesn't really matter as far as I can tell, but you probably will need to follow their scheme and send as text/plain that just happens to be valid JSON.
FYI this has nothing to do with python
GOOD
POST /i/cards/api/v1.json?tweet_id=835553193853607936&card_name=poll2choice_text_only&authenticity_token=3fa4898d1eb9c3970d122438069080f67e2c40a0&forward=true&capi_uri=capi%3A%2F%2Fpassthrough%2F1 HTTP/1.1
Host: twitter.com
Connection: keep-alive
Content-Length: 146
Origin: https://twitter.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Content-Type: text/plain;charset=UTF-8
Accept: */*
Referer: https://twitter.com/i/cards/tfw/v1/835553193853607936?cardname=poll2choice_text_only&autoplay_disabled=true&forward=true&earned=true&lang=en&card_height=114&scribe_context=%7B%22client%22%3A%22web%22%2C%22page%22%3A%22search%22%2C%22section%22%3A%22news%22%2C%22component%22%3A%22tweet%22%7D
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Cookie: guest_id=v1%3A148348519609568388; eu_cn=1; _ga=GA1.2.17648078.1483840185; moments_moment_guide_create_moment_tooltip=true; moments_profile_moments_nav_tooltip_self=true; moments_profile_moments_nav_tooltip_other=true; d=32; mobile_metrics_token=148355421759690664; zrca=5; u=4345167ab07c53b1c1a0b0f28ca3c23fa597e826; lang=en; kdt=wBD2EQ4hAtxU5AgI2pWJhBvaYVh86CwyARordbHz; remember_checked_on=1; twid="u=797054191420276736"; auth_token=2e4444c70ae4faccc4b89b68f268d0ffca2582c6; ct0=052eee2c9ef178e33f39a1b1effb89b9; external_referer="padhuUp37zguUvQ9eLPWcgjfN2q0WSYBeFiMalFf9QUYFrDw6JjXFQ==|0"; m5=off; _mobile_sess=BAh7ByIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7ADoQX2NzcmZfdG9rZW4iJWU5OTIzMGE4NmRmYTkyMDY0ZGIxNTAzMDQ3NzAwM2Yz--3c7ec2a6fcdc8d2e2e23eb3403864c855e95d9cb; external_referer="ziZgIoZIK4nlMKUVLq9KcnBFms0d9TqBqrE/yjvSFlFJR45yIlYF+w==|0"; _twitter_sess=BAh7CiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCL3hjF1aAToMY3NyZl9p%250AZCIlNjZmZTZjNGVmYjVkODlkYzAxYzk1NzBhODIyOWY5ZDM6B2lkIiU4YjM2%250ANmE2OGNiMGMxODdhMzhlYTQ2OTVkZWJmODI4ZjoJdXNlcmwrCQCAV2qQtA8L--9c34f3e72eedf83c6f85531b076460036abb3694
AlexaToolbar-ALX_NS_PH: AlexaToolbar/alx-4.0.1
BAD
POST /i/cards/api/v1.json?tweet_id=835562654030835712&card_name=poll2choice_text_only&authenticity_token=b81b369798790ff7dde3dba3d4ad1b117ec687e8&forward=true&capi_uri=capi%3A%2F%2Fpassthrough%2F1 HTTP/1.1
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
Origin: https://www.twitter.com
Host: www.twitter.com
X-Requested-With: XMLHttpRequest
Cookie: remember_checked_on=1; ct0=d46435d2c2d85abcb4f9fb2c8073f03b; _twitter_sess=BAh7CiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCNyzMXdaAToMY3NyZl9p%250AZCIlYzQ3ZTJlYWViYjkwZmEyOTYzYjZmOWEzN2ExMTZkZWY6B2lkIiVmYWI1%250AYmE2MGNjZjcwZWEzMjA0NjdkMDVkNDM3Y2ZjNzoJdXNlcmwrBxX5Xp8%253D--dc874a43cd229d67910c46349ab308b34c3e4aa9; auth_token=89d44c89919ebe49e61263615f731a11e8c20349; guest_id=v1%3A148805843042642246; twid="u=2673801493"; kdt=PEQqxFONd9KhIWG4hUSiToDB58U5qsxPR3ZJLNRX; lang=en
Content-Length: 178
Content-Type: application/x-www-form-urlencoded -
2017-02-25 at 9:39 PM UTCDon't worry, the auth tokens have expired.
-
2017-02-25 at 9:40 PM UTCwait, so did it work?
-
2017-02-25 at 10:04 PM UTCNope, even with the new one
POST /i/cards/api/v1.json?tweet_id=835538193269215237&card_name=poll2choice_text_only&authenticity_token=8d565ef82d56a9a455389621f5b17e3f6e98aad9&forward=true&capi_uri=capi%3A%2F%2Fpassthrough%2F1 HTTP/1.1
Connection: keep-alive
Accept-Encoding: gzip, deflate, br
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Origin: https://twitter.com
Host: twitter.com
Referer: https://twitter.com/i/cards/tfw/v1/835538193269215237?cardname=poll2choice_text_only&autoplay_disabled=true&forward=true&earned=true&lang=en&card_height=114&scribe_context=%7B%22client%22%3A%22web%22%2C%22page%22%3A%22search%22%2C%22section%22%3A%22news%22%2C%22component%22%3A%22tweet%22%7D
AlexaToolbar-ALX_NS_PH: AlexaToolbar/alx-4.0.1
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Content-Type: text/plain;charset=UTF-8
Cookie: remember_checked_on=1; ct0=11f3b1310a3c890366a358e68641a5c7; _twitter_sess=BAh7CiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCDO%252BS3daAToMY3NyZl9p%250AZCIlMmVjN2QyYzhjMGY4ZmQ2OTIxYjUwM2MwNTU5MTFlZDk6B2lkIiU1ZWY0%250AZWJkMjAxYjFiN2MzYzIwMDczMTIyYmExNDg2ZjoJdXNlcmwrCQCAV2qQtA8L--d9d8542ca65d390ce4e55dc29f7f841dcbafc8fc; auth_token=2ee0094edd29f886e1b2ff111f3ccf3cc9e361d8; guest_id=v1%3A148806013701074387; twid="u=797054191420276736"; kdt=C5Q6OFdZ7EAy3RY1sJAIqpoxa12H4VxmBQzhB1dK; lang=en
Content-Length: 178 -
2017-02-25 at 10:21 PM UTCOK, headers look reasonable, what's the request body? What are the response headers?
-
2017-02-25 at 10:26 PM UTC
POST https://twitter.com/i/cards/api/v1.json?tweet_id=835523028515753986&card_name=poll2choice_text_only&authenticity_token=669c3ced7948a698ccfea2205c2dc51fc4887745&forward=true&capi_uri=capi%3A%2F%2Fpassthrough%2F1 HTTP/1.1
Connection: keep-alive
Accept-Encoding: gzip, deflate, br
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Origin: https://twitter.com
Host: twitter.com
Referer: https://twitter.com/i/cards/tfw/v1/835523028515753986?cardname=poll2choice_text_only&autoplay_disabled=true&forward=true&earned=true&lang=en&card_height=114&scribe_context=%7B%22client%22%3A%22web%22%2C%22page%22%3A%22search%22%2C%22section%22%3A%22news%22%2C%22component%22%3A%22tweet%22%7D
AlexaToolbar-ALX_NS_PH: AlexaToolbar/alx-4.0.1
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Content-Type: text/plain;charset=UTF-8
Cookie: remember_checked_on=1; ct0=90c008ac2c3791405e6fbc3181d55fd9; _twitter_sess=BAh7CiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCFlTX3daAToMY3NyZl9p%250AZCIlZjM5MjE3NzRiOGYzYTExODE2Y2QxMTU0NmY4MWU3ODk6B2lkIiUzNmZm%250AN2U0NzQwN2NkOTE3YjU2YmM2ODVjYzhmNGJkZToJdXNlcmwrCQCAV2qQtA8L--6a40fd8d80ec80939990ae39fc223d3c3b7aa36a; auth_token=78a6b3c4f4119bda8f943c5a4b8f7c858355d8f8; guest_id=v1%3A148806142037727180; twid="u=797054191420276736"; kdt=5xhzgmWWJL6GjzDfnpBWTHGqR2FEqcEgQXXJ5ARY; lang=en
Content-Length: 151
{"twitter:long:original_tweet_id": "835523028515753986", "twitter:string:card_uri": "card://835523028515753986", "twitter:string:selected_choice": "1"}
respone:
HTTP/1.1 200 OK
content-disposition: attachment; filename=json.json
content-length: 123
content-type: application/json;charset=utf-8
date: Sat, 25 Feb 2017 22:24:05 GMT
server: tsa_f
set-cookie: fm=0; Expires=Sat, 25 Feb 2017 22:23:55 GMT; Path=/; Domain=.twitter.com; Secure; HTTPOnly
set-cookie: _twitter_sess=BAh7CiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCFlTX3daAToMY3NyZl9p%250AZCIlZjM5MjE3NzRiOGYzYTExODE2Y2QxMTU0NmY4MWU3ODk6B2lkIiUzNmZm%250AN2U0NzQwN2NkOTE3YjU2YmM2ODVjYzhmNGJkZToJdXNlcmwrCQCAV2qQtA8L--6a40fd8d80ec80939990ae39fc223d3c3b7aa36a; Path=/; Domain=.twitter.com; Secure; HTTPOnly
strict-transport-security: max-age=631138519
x-connection-hash: cb5f9245ed42d3ba5c10c315192d81a4
x-response-time: 133
x-transaction: 008fc5c200506a24
x-tsa-request-body-time: 1
{
"successful" : false,
"response" : {
"cardName" : "poll2choice_text_only",
"message" : "Not available."
}
} -
2017-02-25 at 10:32 PM UTCLanny found the problem, poll ID is different from the tweet ID :) How can I use bs to get these 2 values:
data-card-name="poll2choice_text_only"
data-card-url="card://835523027689435143"
from:
<div class="js-macaw-cards-iframe-container initial-card-height card-type-poll2choice_text_only"
data-src="/i/cards/tfw/v1/835523028515753986?cardname=poll2choice_text_only&autoplay_disabled=true&earned=true&lang=en"
data-card-name="poll2choice_text_only"
data-card-url="card://835523027689435143"
data-publisher-id=""
data-creator-id=""
data-amplify-content-id=""
data-amplify-playlist-url=""
data-full-card-iframe-url="/i/cards/tfw/v1/835523028515753986?cardname=poll2choice_text_only&autoplay_disabled=true&earned=true&lang=en"
data-has-autoplayable-media="false">
</div>
I've tried:
#card_name = code.find("div", {"data-card-name"})
but that doesnt work
Post last edited by RichardBurnish at 2017-02-25T22:38:14.388504+00:00 -
2017-02-25 at 11:24 PM UTC.find takes arbitrary kwargs, which is interpreted as search by attribute. If you pass a bool it just tests for attribute existence. The trick is that data-card-name isn't a valid identifier in python so you need to use dict destructuring to make this work like so:
code.find("div", **{'data-card-name': True})
Looks wierd right? A little language trivia. This:
func(arg1, **{'some_keyword': 42})
is always equivalent to:
func(arg1, some_keyword=42)
The ** syntax on the caller side says "unpack the keys of this dict-like and pass them as keywords to this function" in the same way * does list splaying. The former style can express slightly more keywords than the latter because you can express every string as a key in a dict but not as a parameter name. Kind of language wankery, I think this is the first time I've ever actually had to do this in python, but kind of a fun fact.The following users say it would be alright if the author of this post didn't die in a fire! -
2017-02-25 at 11:28 PM UTCthis was fun, got to bust out ncat for the first time in a while. I feel so leekThe following users say it would be alright if the author of this post didn't die in a fire!
-
2017-02-25 at 11:58 PM UTC
Originally posted by Lanny .find takes arbitrary kwargs, which is interpreted as search by attribute. If you pass a bool it just tests for attribute existence. The trick is that data-card-name isn't a valid identifier in python so you need to use dict destructuring to make this work like so:
code.find("div", **{'data-card-name': True})
Looks wierd right? A little language trivia. This:
func(arg1, **{'some_keyword': 42})
is always equivalent to:
func(arg1, some_keyword=42)
The ** syntax on the caller side says "unpack the keys of this dict-like and pass them as keywords to this function" in the same way * does list splaying. The former style can express slightly more keywords than the latter because you can express every string as a key in a dict but not as a parameter name. Kind of language wankery, I think this is the first time I've ever actually had to do this in python, but kind of a fun fact.
BOOM!!! thanks, bud :)
-
2017-02-26 at 1:38 PM UTC
