What is the order in which classes are instantiated in PythonGenerally speaking, there are class variables and methods in a class. For example, we define a class named A. class A(): bar = “my lover love me” def __init__(self, name): print(‘A的class’ ,self.__class__, name) We have defined a class variable bar and a construction method in ..
Category : python
Introduction to generator In python, an object containing the yield keyword is a generator that is invoked every timenextThe method is executed untilyieldThe following statement, and then returnyieldThe execution results of the following code blocks. In fact, you can also call the send methodThe following is an example for easy understanding. Next method def foo(): ..
Design pattern learning 1. Decorator mode This is introduced in my notes. 2. Factory model author: liibntime:2018-11-6Not completed, to be updated 3. singleton pattern Ensure that at most one instance of an object exists. If there are multiple statements that create an instance, only the first one will actually create an instance, and the following ..
Python Multiprocess, Global Variable Not Updated Problem: Sub-process global variables are not updated as expected, resulting in data not being written to disk. The process pool must be in theif __name__ == ‘__main__’The following is usedReproduction: see the experimental code.Environment: centos7, python3.7Reason: of the subprocess__name__Attribute value is not’__main__’Solution: The resources needed should be inif __name__ ..
Python counts the number of all files under the specified folder I have always had this requirement, but I used to write it recursively, feeling unfriendly to the occupation of resources, and python’s maximum recursion depth is not more than 1,000, so I changed it and implemented it by breadth-first traversal.A total of 24 folders ..
Oozie creates workflow The execution command of the workflow refers to the blog:https://www.jianshu.com/p/6cb3a4b78556, you can also typeoozie helpView help Manually configure oozie’s workflow Job.properties file, which stores some parameters that may be used in workflow.xml filejob.properties # 注意变量名不要包含特殊字符，否则在 spark 中会出现无法解析变量名的问题 # oozie.wf.application.path的路径必须在hdfs上，因为整个集群要访问 nameNode=hdfs://txz-data0:9820 resourceManager=txz-data0:8032 oozie.use.system.libpath=true oozie.libpath=${nameNode}/share/lib/spark2/jars/,${nameNode}/share/lib/spark2/python/lib/,${nameNode}/share/lib/spark2/hive-site.xml oozie.wf.application.path=${nameNode}/workflow/data-factory/download_report_voice_and_upload/Workflow oozie.action.sharelib.for.spark=spark2 archive=${nameNode}/envs/py3.tar.gz#py # 如果 dryrun 为 true，表示只是测试当前的 workflow，并不具体记录相应 ..
Python3.7 dictionary is orderly Old structure The dictionary structure before python3.7, the classic and crude hash table implementation, in this way every expansion and contraction of the hash table may cause the hash value to change. Before and after the hash table capacity update, the relative order between its keys will change, so the elements ..
Climbing to car home Used Car Product Bank Project address:https://github.com/go-crawler … Target Recently, people often mention car home in their ears and wonder about the price of used cars in China. Therefore, the target site for this time isHome of the carThe second-hand car products warehouse Analyze target source: There are 24 articles in a ..
Learn about Golang’s market situation Project address:https://github.com/go-crawler … If it is helpful to you, welcome Star and give the article a praise so that more people can see it:) Target Golang is already a part of the work. I want everyone to know about the market situation of Golang, and I also want more people ..
Fake-useragent of Go Some websites will jump to different (PC, M) websites according to the different User-Agent, and will give different prompts according to different versions, etc. The change of User-Agent is the basic posture in the crawler. If you use Go to write a web crawler or need to simulate a User-Agent, will it ..
About elastalert Elastalert is an elasticsearch-based alert service produced by yelp and written in python. The overall idea is still based on polling method. If rules are applied, frequency, spike, flatline, blacklist/whitelist, any and change are built in. In case of alarm, Email, HipChat, Slack, Telegram, etc. are provided. dockerfile # Elastalert Docker image running ..
example #coding=UTF-8 import sys import MySQLdb import time reload(sys) sys.setdefaultencoding(‘utf-8’) def connectDemo(): return MySQLdb.Connection(“127.0.0.1″,”root”,”root”,”demo”,3306,charset=”utf8″) if __name__ == ‘__main__’: begin=time.time() conn=connectDemo() cursor = conn.cursor() sql=””” show tables “”” count = cursor.execute(sql) rows = cursor.fetchall() cursor.close() conn.close() print “========demo库共:%s 张表============” % (count) print ‘耗时:%s 秒’ % (time.tim..