It’s time to write the year-end summary again. At this time of the year, my mind is always churning, because I haven’t reflected and summarized it for too long. I only summarize it once a year. It’s really a little long. I owe too much and it’s a little difficult to sort it out. This is still a summary of work/technology.
In 2017, he led a small project from front to back. the technical system used openresty+springboot/cloud+postgres+statsd+graphite+grafana+sentry+elasticsearch+kafka+doc+kubernetes. This year is still a continuation of the above technical framework for in-depth:
- docker & kubernetes(
Stay in the use stage, considering the maturity of cloud platform technology, assuming you go out and start your own business, you don’t feel you need to build kubernetes. You can directly use the basic settings of kubernetes such as Aliyun and Tengxunyun, so there is no in-depth study.
- spring cloud(
Practice and Extension Levels)
This year is a year of in-depth practice of a spring cloud, but the production is still in the Camden.SR6 version, the current GA version is already Edgware, and the pre version has reached Finchley M4. Moreover, if the Camden version is migrated to the above version, it seems that there are some incompatibilities or changes, which are a little tired of follow-up. In addition, the main reason is that the production has already run stably, so there is no strength to upgrade it again, and stability is the main factor. When you do your next project, consider using the latest version directly. However, as Servimesh matures, the distributed set is further solidified into the basic settings, and the potential advantages of spring cloud in the future are less obvious. Moreover, with the popularity of go language and the enrichment of related ecology, many high concurrent services are gradually replacing java with go. If it is not for applications with strong business characteristics, the advantages of java may not be so great. Just like …Java may no longer be the preferred development language for e-commerceThis article said.
- spring security(
Level of usage and expansion)
This year, I went to see spring security and spring security oauth2 again. I felt a little bit less, lacked to implement a whole set of technologies to replace the existing authority control system, did not apply them on a large scale, and always felt that I did not have in-depth understanding. In addition, we began to attach importance to the knowledge of system security and are studying it.
Use and understand the basic principles)
The kafka version is too miscellaneous. Some companies use 0.8, some use 0.9, some use 0.10, and in 17 years they launched 0.11. Then the 1.0 version came out. For beginners, they felt a little caught off guard. Once they entered the technology, it was like the sea. It was really a sea of learning, so they had to pick and see. I mainly studied some details of the use and implementation of version 0.8, and also studied some differences between version 0.9 and version 0.8, as well as kafka stream, which was mainly introduced this year, and I may further understand it based on version 1.0.
- redis / postgres / mongo / elasticsearch(
These have not been used much this year, so they will be used once.
Talking about relational databases, I have not been interested in SQL and have not conducted in-depth research. I have seen too much object-oriented view. I have also seen too many examples of complex business requirements writing complex SQL and then hanging up the system. I always think that other ways of implementation can replace complex SQL and have better performance. The emergence of NOSQL and many non-transactional queries can basically be realized by synchronizing the structured/redundant data according to the query requirements into NOSQL.
The other several NOSQL, if you want to rank, it is estimated that redis will have priority, after all, it is too widely used, and it is estimated that you will have to learn Redis further next year. I don’t like mongo very much. I don’t feel as mature as elasticsearch for things like fragment duplication. here, the head should be ranked, and elasticsearch should be ranked in front of mongo. In the words of elasticsearch, I had thought about studying deeply before, but the version changed too fast. Previously it was version 1.x, but now it is version 6. x. I want to cry, especially the use of java api, and I am tired of studying. Another thing is that the cost of learning query syntax for mongo and elasticsearch is too high. elasticsearch is better. most scenes with ES SQL plug-ins can be queried with SQL, which is still rather awkward. mongo has no SQL plug-ins to check, so it is even more reluctant to learn mongo.
As for message queue, Rabbit mq is the most dazzling among the traditional MQ, but with the appearance of kafka and rocketmq, it is somewhat submerged. However, it is still worth learning and needs further practice. Focus on the open-messaging specification, starting with this.
To get a little used to)
When writing lua for the first time, I felt it was very difficult to debug. Without idea to prompt me, I always restart, request and check the log for verification. Later, I wrote too much and got a little used to it. I felt that it was OK. It was mainly nginx second restart, which was very cool. Further research is needed. However, with the appearance of go, there are gateways for subsequent go, such asjanus, visual can spell a spell.
- arch / system design(
In terms of system architecture, I feel that I am still at the entry level. Further study and improvement are needed, not only in terms of technical architecture, but also in terms of business architecture.
After experiencing the back-end technology architecture and implementation from zero to one, the expectation is to initially have the ability to start a business and pull up a complete set of back-end technology architecture and implementation.
The number of visits to the system is too low to expose problems. Some technology implementations are different in the number of visits of millions, tens of millions and hundreds of millions. Moreover, high concurrency scenarios will expose problems that are not well achieved by some technologies. This is the hard injury of this project done this year.
Achievement of goals
- Continue to blog every day, continue to improve sg reputation (
This was completed, breaking 1000 reputation in 16 years and 6000 reputation in 17 years.
- Answer stackoverflow more questions (
Not up to standard)
This is not very good, with 255 prestige in 16 years and 457 prestige in 17 years, up 24,052 overall, but ranking 98894.
- Every weekend the technology resumes (
This is almost unfinished, is the most deadly, 18 years to strictly grasp this
- I still need to continue running in summer, hoping to learn how to swim (
I didn’t learn swimming, but I did insist on running. Apart from rain and working overtime, I was too tired to run. I ran every Monday. I ran more than 80 kilometers in 17 years. At the peak, I could run 4km at a time, but it didn’t last long. I hope to run 5km more times in 18 years.
- Less overtime, more promotion time and project management ability (
Not up to standard)
Habitual overtime, feel the body will be working overtime gradually work overtime is broken, the time planning is not good, several times research problem research for several hours to one or two to sleep, too much damage to the body.
Learning Plan 2018:
- Key study
System architecture (
CASE MOCK), in-depth system security (
OWASP), deep distributed cache (
redisIn-depth study of open-tracing (
kafka), in-depth study of nginx/openresty
Non-technical, the goal is to study financial investment and its combination with technology.
- Frontier Technology Exploration
Service mesh and spring cloud may be in the limelight, but they are only limited to the java field. However, service mesh is not limited to languages. Further solidifying the distributed architecture at the bottom is another technological change.
I’ve always wanted to learn AI, but I didn’t find the motivation to start. I guess I’ll start with some theories of machine learning and find some practical points to learn while practicing.
- Continue to sg every day, sg reputation exceeds 10000
- Stackoverflow reputation exceeds 1000
- Weekly technical resume /CASE STUDY (
Complete let it crash series)
- Complete at least one investment article per week (
Gradually deepen financial business knowledge)
- The number of annual running kilometers reaches 100+ kilometers.
- Complete the study of system security, distributed caching, open-tracing and nginx/openresty.
- Research and practice of service mesh
- Financial Technology Related Attempts
- Practice of High Concurrency and Multiple Opportunities
- More Research on Excellent Open Source Project Architecture/Code
- Write More Open Source Projects
docker / kubernetes
- Docker’s Network Configuration
- Docker modifies the number of open files
- Docker-machine specifies the number of cpus
- Apt-install processing continue in dockerfile
- Docker runs prometheus
- Docker runs jenkins
- Docker runs oracle11g
- Docker installs tesseract
- Dockerfile installs maven
- Docker Builds mongo Copy Set
- Docker runs consolicluster
- Docker runs drill 1.10.0
- Docker java8 installs JCE
- Ubuntu doc manually installs java8
- Docker runs seyren
- Docker-compose runs sentry.
- Docker runs mywebsql
- Docker runs storm and wordcount instances
- Docker runs kafka manager
- Docker 360 pika
- Ubuntu installs chrome and firefox.
- Kompose artifact use
- Type of service for kubernetes
- RollingUpdate using kubernetes’ deployment
- Gracefully Close nginx in kubernetes
- Mac installs kubernetes and runs echoserver.
- Kubernetes installs heapster, influxdb and grafana
- Kubernetes deploys rabbitmq cluster
http / tcp / network / io
- Several Properties of cookie
- Cache-Control parameters
- Netstat and tcp status
- ListenOverflows and other parameters about netstat
- Tcp’s Semi-connected and Full-connected Queues
- ListenOverflows and other parameters about netstat
- Asynchronous IO
- IO intensive thread pool size setting
- What exactly is netty’s Sharable for
- Webmagic trial scalpel
- Httpclient parameter configuration
- Apache’s HttpClient’s default retry mechanism
- Httpclient access https
- RestTemplate access https
- Converters configuration for RestTemplate
- RestTemplate sets the generic header
- RestTemplate gets the contentType of the file
- Springboot configuration okhttp3
nginx / openresty
- Talking about Several Common Anomalies of nginx
- Talk about nginx Error 499
- Talk about nginx and tomcat’s 5xx
- Nginx lua reset request parameter and constant memo
- Nginx Domain Name Configures 301 Jumps for Non-80 Ports
- Nginx lua instruction execution order
- Considerations for Using nginx capture
- Upstream exception for nginx
- Openresty installation and use of LuaXml
- Proxy_redirect for nginx
- Tomcat boot process for springboot
- Tomcat thread pool policy
- Tomcat’s acceptCount and maxConnections
- Tomcat’s acceptCount, maxThreads, connectionTimeout parameter adjustment
- Springboot adds metrics to tomcat
- Tomcat jdbc pool advanced configuration
- Borrow and return of tomcat jdbc pool
- Tomcat’s jdbc Connection Pool PoolExhaustedException
- Analysis of suspect and abandon Operations of tomcat jdbc Connection Pool
- Interpretation of tomcat jdbc SlowQueryReport Implementation
- An Analysis of tomcat jdbc’s ResetAbandonedTimer
- Springboot visits tomcat’s SlowQueryReport
- How does tomcat close response’s outputStream
java / jvm
- AllMatch for Java8
- Java9 Series (1) Installation and jshell Use
- Java9 Series (2) docker Runs java9
- Java rewrites equals and hashcode methods
- Threeten-extra instance
- Java serializes enumerations into objects
- Java computation set intersection union set
- Java computation set symmetry difference
- Java character full-width half-width conversion
- Format time using prettyTime
- How does java get hdd serial number
- Execution thread of CompletableFuture
- CommonPool Related Parameter Configuration of ForkJoinPool
- Java cancel thread instance
- CompletableFuture Usage Instance for java8
- Thread Pool Work Stealing Instances
- Fastjson gets the unknown field
- Jni loading of rocksdb
- Md5 encryption using common-codec
- Lombok’s builder Sets Default Values
- Talking about Parameter Verification of lombok Construction Mode
- Talk about the querydsl generation class.
- Remember a toJSONString exception
- Resolve SAXNotRecognizedException
- Java calculates object size
- Use SXSSFWorkbook to export excel
- TransferQueue instance
- Use case of Deque
- SynchronousQueue usage instance
- Some Methods to Reduce Competitive Locks in java
- ArrayBlockingQueue and LinkedBlockingQueue
- ConcurrentLinkedQueue usage instance
- Analysis of timeout principle of futureTask
- JIT compiler version
- Jvm parameter view
- GC log parameters
- Gc thread count calculation formula for jvm
- Jvm heap size configuration recommendations
- CMS collector
- Jvm hierarchical compilation level
- Jvm code cache
- Resolve jd-gui Flashover under Sierra
- Talk about PermGen and Metaspace of jvm
image / pdf
- Talk about alpha values in Color
- Talk about the bug that ImageIO used argb to operate jpg.
- Use BufferedImage for gradient operation
- Talk about the timeout parameter of selenium’s webdriver.
- Talk about the construction of Selenium’s different webdriver.
- Resolve cannot be applied in WebDriverWait
- Talk about the optimization measures of phantomjs js.
- Phanbedder use case
- Java compresses and resize pictures.
- Use imgscalr for picture manipulation
- Using tesseract to identify graphic verification code
- Image verification code identification using asprise
- Using zxing to Generate and Identify Two-dimensional Codes
- Talk about zxing’s qrcode
- Talk about qrcode’s detect position.
- Remember an OfficeException
- Improvement analysis of version 4.1.0 of jodconverter4.1.0
- Use pdfdom to convert pdfs to html
- Pdfbox to image
- Use tika to convert pdf to html
- SpringMVC data binding instance
- How does spring mvc calculate best _ matching _ pattern _ attribute
- Comparison of Several Interceptors in spring mvc
- Springmvc continues to output text to web pages.
- Springboot custom 404 error message
- Send html mail using spring mail
- SpringBoot profile date attribute conversion instance
- Spring gets information about custom annotations in the controller method
- The PathVariable of springboot receives parameter values with dot numbers.
- File download is written by chunk
- Resolve incompatibility between metrics-spring and springboot1.4
- Springboot dynamically loads sigar
- Springboot dynamically loads native class library
- Springboot different servlet containers benchmark
- Springboot runtime loads statsd-jvm-profiler
- HealthAggregator for springboot
- Close FAIL_ON_EMPTY_BEANS for spring boot jackson
- Springboot asynchronous mvc uses the correct posture of threadlocal.
- Springboot sets the timeout for RestTemplate.
- ApplicationReadyEvent for springboot
- Event-driven development using reactor eventbus
- Spring event publishing and monitoring instance
- How to Access RequestContextHolder in async Thread
- Talk about spring’s async comments
- Talk about the spring hosting of TaskExecutor.
- DiskSpaceHealthIndicator of springboot
- Extending tomcat’s executor in springboot
- Resolve the problem of springboot reading files in jar packages
- ConfigurationProperties for springboot uses
- Spring boot admin uses consolidiscovery.
- Use cached’s wrapper class to read the request response content
- Method of repeatedly consuming input stream
- Springboot Environment injection exception
- Reactor custom RejectedExecutionHandler
- Logback output log to sentry
- Logback Log Increase Line Number
- Job design of activiti
- Quartz sets misfire policy
- Quartz’s api and Database Table Mapping Analysis
- Use springboot to construct quartz’s rest api.
- Talk about quartz’s Scheduling and Performance
- Springboot integration quartz2.3.0
- Springboot integrated mqtt
- Getting Started with esper in springboot
- Zuul modification request url
- Fix zuul cross-domain configuration exception
- Zuul Custom SimpleHostRoutingFilter
- Some Common Exceptions of zuul Forwarding
- Order summary of ZuulFilter
- Zuul carries out rate limit
- Zuul adds request header
- Zuulfilter gets id of proxy
- Zuul’s forward error
- Zuul modify request response body
- Zuul modified request query param
- Zuul dynamic route loading configuration
- Zuul file size limit
- Zuul forwarding Set-Cookie loss problem
- SpringCloud runtime refreshes data source related configurations
- Talk about ConfigClientWatch of Spring Cloud Config.
- Remember a hystrix exception.
- Hystrix maxconcurrentconnections reacted exception
- Retry and timeout settings for feigninclient
- Feignclient sets hystrix parameters
- Interception by feignclient
- On the abnormality of feigninclient triggering fusing
- Spring cloud feign uses okhttp3
- Spring cloud feign uploads files
- Spring cloud eureka parameter configuration
- Understanding eureka’s Self-protection Mechanism
- How does EurekaClient update registration information
- How does eureka eliminate instances
- Eureka’s Surprise Group Effect
- RateLimiter for eureka
- Spring cloud atlas uses
- Use springboot admin in spring cloud
- ribbon static server list
- A pit on the ribbonClient configuration
- Ribbon Sets url Level Timeout
- Ribbon personalization parameter settings
- ServerListRefreshInterval for ribbon
- The ribbon uses eureka’s meta for dynamic routing.
- Springcloud canary routing scheme based on ribbon
- Consul’s server mode enables ui
- Springcloud file upload file name garbled
security / spring security / oauth2
- Java encryption and decryption instance
- Generate totp using aerogear
- Talk about QR Code Login
- Talk about directory traversal attack
- Talk about sessionfixations
- Talk about springboot session timeout parameter setting.
- Spring security dynamically configures url permissions
- Scheme 2 for spring security to dynamically configure permissions
- Spring security runtime configuration ignore url
- Spring security login-free dynamic configuration scheme 2
- Spring securityax login and return
- Spring security custom AuthenticationEntryPoint
- Spring security customization guide
- Talk about spring security’s role hierarchy.
- Talk about securitycontexterpersistencipherer
- Talk about FilterSecurityInterceptor
- Talk about spring security’s Account Locking
- Talk about the use of SwitchUserFilter
- SwitchUserFilter Source Code Analysis
- Talk about permitAll and webIgnore of spring security.
- List all SecurityFilterChain of spring security
- Spring security filter gets the requested urlpattern
- Solve the problem of repeated execution of spring security custom filter.
- Summary of oauth2 Four Authorization Methods
- Spring securityouth2client _ credentials mode
- Spring securityouth2password authorization mode
- Spring securityouth2 authorizationcode mode
- Spring securityouth2 implicit mode
- Use spring-security-oauth2 as client implementation.
- Spring securityouth2 refresh token
- Talk about the password authentication of spring securityouth2.
- Talk about the certification of several endpoint of spring security oauth2
- Spring securityouth2 uses redis to store token
- Spring securityouth2 allowformauthenticationforclients principle analysis
jdbc / redis / elasticsearch / mongo
- Talk about concurrent update of sql
- Summary of Drag Database Design Scheme
- Javax’s PrePersist annotation
- Jdbc connection pg database timeout
- Remember an exception with too many pg connections
- Using hibernate to Generate Table Building Statements
- Talk about the implementation of hibernate’s hbm2ddl
- Spring data jpa data change audit
- Javer’s Table Structure Design
- Use envers to Record Data Changes
- Talk about RowMapper of spring jdbc.
- Spring-data-jpa setting fetchsize
- Auditor settings for spring-data-jpa
- Spring-data-jpa soft deletion scheme
- Spring data jpa uses native sql instance
- Jpa stores byte to postgresql
- Mybatis spring boot starter configuration
- Jpa postgresql uses uuid as the primary key
- Mongo View Database Space Size
- Redis monitoring command
- Talk about several daos of jesque.
- Talk about jesque’s data structure in redis
- Talk about jesque’s event mechanism.
- Talk about jesque’s WorkerPool and workerpool.
- Springboot uses jest to operate elasticsearch.
- Fix jest’s Connection is still allocated exception
- Mongo’s geo query
- Talk about spring’s Integration with kafka
- Springboot integrated akka
- Spring cloud stream kafka instance
- Spring-cloud-stream-binder-kafka attribute configuration
- Kafka0.8 Producer Example
- Kafka0.8 Consumer Example
- Analysis of kafka0.8 Producer Configuration Parameters
- Kafka0.8 Producer Exception Handling
- Talking about kafka’s partition Allocation
- Talk about the consumer Timeout Property of kafka0.8
- Let’s talk about the partition of topic in kafka0.8 and topicCountMap.
- Talk about kafka consumer offset lag increase exception
- Talking about the monitoring of kafka consumer offset lag
- Talk about the MaxLag index of kafka 0.8 ConsumerFetcherManager.
- Talk about kafka client chunkQueue and MaxLag values
- Kafka0.10client use case
- Talk about retry of spring kafka
- Talk about AckMode of spring for kafka
- Spring for kafka automatic configuration and configuration properties
- Custom spring kafka consumer thread pool
- Talk about kafka’s group coordinator
- Talk about the auto commit of kafka client
- Talk about spring for kafka’s Packaging and Integration of producer
- Talk about spring for kafka’s Packaging and Integration of consumer
- Join instance of kafka streams
- Customize processor for kafka streams
- Kafka stream errorlog alarm instance
- Kafka stream word count instance
- Spring boot admin slack alert
- Springboot reads metrics from dropwizard.
- Springboot outputs metrics to prometheus
- Output hystrix indicator to dropwizard metrics
- The Framework and Persistence of Prometheus
- Monitoring java Applications in docker through jmx
- Pcp Monitoring docker Application of spring boot
- Docker runs badjs
- Use the alert function of grafana4.
- Read /proc/net/dev in java
- Curl records response time
- Calculation of Process cpu Usage
- Java gets stat for the specified process
- Several indicators of statsd timer
- Several function on graphite
- On the HistogramMetric of codahale
- Grafana uses mysql storage
- Size Control of SlidingTimeWindowReservoir
- Grafana high availability architecture
- Sentry8.15 Disable Registration
arch / design / data structure
- ArchiMate hierarchy concept
- How to Talk about partition
- Talk about replication
- Overview of distributed id generation scheme
- Talk about base62 and tinyURL
- Design of High Performance SPSC Lockless Queue
- Cache Aside mode for caching
- Introduction to JCTools
- Details of HashedWheelTimer Algorithm
- Brief Introduction of lamport Bakery Algorithm
- Use ratelimitj to limit the flow of apis.
- Random Load Balancing Routing Algorithm Based on Weight
- Java version of hyperloglog
- Talk about jump consistent hash
- Simple implementation of bloomfilter
- Use the appropriate data structure to count the number of words
- Summary of Sensitive Word Detection Algorithms
- Trie tree usage instance
- Talk about leak detection of GenericObjectPool
- Analysis of GenericObjectPool Parameters
- Shedlock source code analysis
- Use shedlock to lock spring schedule.
mac /linux /app build
- Method for dynamically generating ios plist
- Ios package error: userinteraction is notallowedd
- zsh: you have running jobs
- Replace text with sed
- mac unable to get application ASN from launchservicesd
- Tesseract3.04.01_2 Sample Training on mac
- Mac installation consul
- Shell Get File Size and checksum
- Sign and align commands for apk files
- Simplify parameterizations with JUnitParams
- How to Pressure Test Login Interface in Batch
- Use jmh for Micro Benchmarking
- Several Key Points of Performance Test
- Delivery Pipeline
- Use postman as rest api automated testing tool.
- Batch execution newman
- Jenkins’ bash shell uses boolean values.
- Jenkins Push Mirror to Private docker-registry
- Jenkins Builds docker Mirror
- Jenkins Integrated java Static Code Analysis Tool
- Sonar custom rule
- Jenkins integrated sonarqube
- Deploying Local jar to maven Private Service Warehouse
- Mac installation gitlab-ci-multi-runner running sonar
- Maven warehouse jar package publishing guide
- Maven loads external dependency packages
- Maven’s test uses main’s resources.