Research on supply and demand characteristics of big data science industry based on multi-dimensional analysis

Gaorui Zhang; Huiqin Sun; Xiaohong Wang

doi:10.54097/wyg0fh69

Authors

Gaorui Zhang
Huiqin Sun
Xiaohong Wang

DOI:

https://doi.org/10.54097/wyg0fh69

Keywords:

Crawlers, Data analysis, Data visualization, Data analysts

Abstract

The present study aims to investigate the employment prospects of Data Science and Big Data Technology majors. To this end, a web crawler system has been constructed using Python technology. The software extracts data on "Data Analyst" positions from a recruitment website, performs operations such as data structuring, duplicate removal, salary outlier handling, and standardization of educational requirements. The integration of descriptive statistics and visualization techniques facilitates the establishment of a comprehensive database, encompassing variables such as salary, geographical location, educational attainment, and skillset. Empirical analysis reveals significant regional disparities in salary for data analyst positions, with Beijing, Shanghai, and Shenzhen averaging 25-35K RMB monthly—30%-50% higher than other cities. A positive correlation is evident between educational attainment and salary, with doctoral degree holders earning approximately 1.8-2.2 times the average monthly salary of bachelor's degree holders. It is evident that Python, SQL and Tableau are the skills most frequently mentioned among the skill requirements, with percentages of 95%, 92% and 82%, respectively. The findings of this study provide data-driven insights with regard to the development of academic programs and the planning of careers.

Downloads

Download data is not yet available.

References

[1] Yang Lu. Problems and suggestions analysis of network recruitment in colleges and universities in the era of big data[J]. China Management Informatization,2021,24(19):121-123. DOI:CNKI:SUN:GLXZ.0.2021-19-054.

[2] Shugui Zhang. Design and implementation of Hadoop-based big data platform for smart job analysis[J]. Information and Computer (Theoretical Edition),2024,36(05):112-114+118. DOI:CNKI:SUN:XXDL.0.2024-05-034.

[3] H. Zhang, B. An, J.F. Zhang. Crawling and analyzing the recruitment data of big data professionals based on Python[J]. Journal of Taiyuan City Vocational and Technical College, 2025, (10):76-78.DOI:10.16227/j.cnki.tycs.2025.0619.

[4] Tan YJ. Research on data de-duplication technology in data backup system[D]. Huazhong University of Science and Technology,2012.

[5] Cheng Kaiming. Review of theories and methods of statistical data preprocessing[J]. Statistics and Information Forum,2007, (06):98-103.DOI:CNKI:SUN:TJLT.0.2007-06-020.

[6] H.P. Ding,R.J. Ji,C.C. Zhao,et al. Recruitment data crawling and visualization analysis based on Python[J]. Computer Programming Skills and Maintenance,2025, (08):103-105. DOI:10.16184/j.cnki.comprg.2025.08.017.

[7] Ren Lei,Du Yi,Ma Shuai,et al. An overview of big data visual analytics[J]. Journal of Software,2014,25(09):1909-1936. DOI:10.13328/j.cnki.jos.004645.

[8] Focusing on Digital Economy and Employment--China's Ninety-Eight Human Capital Forum in Xiamen[J]. China Employment, 2022, (10):7-8.DOI:CNKI:SUN:GGJX.0.2022-10-003.