I. overview of steps
A. Simulated login to the school elective system (login to http://xk.suibe.edu.cn/xsxk/login.xk using Selenium library)
B. after obtaining the cookie, it is passed into the session of requests. (Reference Blog: https://blog.csdn.net/big__v/article/details/78151940)
C. Submit the serial number of the course selection to http://xk.suibe.edu.cn/xsxk/xkOper.xk using post in the requests library
D. completion of elective courses
II. Browser Screenshot During Manual Login (Chrmoe)
1. request headers when logging in (here is a cookie where errors will occur later. )
Post at login
III. Overview of errors
1. If you manually log in with the browser and copy the cookie and directly enter the headers inside of get, the get command can be successfully completed and you will be prompted to select the course successfully.
2. If the step of hand-operated browser login is replaced by Selenium simulated login and cookies are directly passed into the session of requests, another attempt at gett will report "illegal entry".
Iv. problem elimination
1. The second step get speculation has no problem, because the cookie in the copy browser directly enters the headers inside of get, and the post command can be successfully completed.
2. It is not a cookie format problem, because the format problem will cause the error "login expired" instead of "illegal login".
V. problem speculation
- Guess cookie Encrypted
- Guess there are other anti-crawler methods
- Guess cookie has timeliness
The complete code is as follows
# -*- coding:utf-8 -*-
from selenium import webdriver
import requests
from selenium.webdriver.support.wait import WebDriverWait
# 添加选课序号
codes = []
while True:
code = raw_input(u'请输入选课序号(输入回车结束):')
if code != '':
codes.append(code)
else:
break
# 开始登录
driver = webdriver.Chrome()
url = "http://xk.suibe.edu.cn/xsxk/login.xk"
s = requests.session()
while True:
driver.get(url)
name_input = driver.find_element_by_id('username') # 找到用户名的框框
pass_input = driver.find_element_by_id('password') # 找到输入密码的框框
login_button = driver.find_element_by_xpath('//*[@id="loginForm"]/table/tbody/tr[4]/td[2]/input[1]')
name_input.clear()
name_input.send_keys('1*******') # 填写用户名
pass_input.clear()
pass_input.send_keys('********') # 填写密码
WebDriverWait(driver, 300000000).until_not(lambda x: x.find_element_by_id("verifyCode").is_displayed()) # 等待直到登录成功
if u'夜大学' in driver.page_source:
print u"登录成功!"
selenium_cookies = driver.get_cookies() # 把selenium获取的cookies保存到变量,备用。
# print(selenium_cookies)
driver.close()
break
else:
driver.close()
# 处理cookie
s = requests.Session()
for i in selenium_cookies:
requests.utils.add_dict_to_cookiejar(s.cookies, {i['name']: i['value']})
# 以下用post提交选课
for code in codes:
info = {'method': 'handleQxgxk',
'jxbid': '201820191' + code,
'glJxbid': '',
'xyjc': ''}
r = s.get('http://xk.suibe.edu.cn/xsxk/xkOper.xk', params=info)
if 'false' in r.text:
print '选课序号:%s,选课失败。' % code, r.text
else:
print '选课序号:%s,选课成功。' % code
The problem has been solved-it is a problem existing in the second verification (I don’t know what it is called, let’s call it that, if there is anything wrong, please point it out in your comments. )
At first, the speculation was that there was something wrong with the coding-after all, python2.7 coding is really a bit of a headache. After all kinds of debugging, it is not a coding problem.
Continue to analyze the browser from scratch and find such a get, as shown in figure:
Guess this is “secondary certification”.
Decisively add the following code to the code:check = s.get('http://xk.suibe.edu.cn/xsxk/xkjs.xk?pyfaid=04265&jxqdm=2&data-frameid=main&data-timer=2000&data-proxy=proxy.xk', headers=headers)
And then it worked.