Parallel Algorithms for Text Sentiment Analysis Based on Deep Learning

ZHAI Donghai; HOU Jialin; LIU Yue

doi:10.3969/j.issn.0258-2724.20160948

Volume 54 Issue 3

Jun. 2019

Turn off MathJax

Article Contents

Abstract

References

Journal of Southwest Jiaotong University > 2019 > 54(3): 647-654.

CHEN Weirong, YU Jin, LI Qi, PU Yuchen, YANG Hanqing, HAN Ying. Balanced Current Control Method for Virtual Synchronous Generator in Electro-Hydrogen Multi-Energy Complementary Microgrid[J]. Journal of Southwest Jiaotong University, 2019, 54(6): 1323-1331. doi: 10.3969/j.issn.0258-2724.20180860

Citation:

ZHAI Donghai, HOU Jialin, LIU Yue. Parallel Algorithms for Text Sentiment Analysis Based on Deep Learning[J]. Journal of Southwest Jiaotong University, 2019, 54(3): 647-654. doi: 10.3969/j.issn.0258-2724.20160948

Citation:

PDF( 1112 KB)

Parallel Algorithms for Text Sentiment Analysis Based on Deep Learning

doi: 10.3969/j.issn.0258-2724.20160948

Received Date: 26 Nov 2016
Rev Recd Date: 27 Nov 2018

Available Online: 11 Jan 2019

Publish Date: 01 Jun 2019

Abstract

Abstract

In the case of big training set and test set, based on semi-supervised auto encoder (Semi-Supervised RAE), the text sentiment analysis algorithm is accompanied by slow training rate and output rate of test results. To solve these problems, the corresponding parallel algorithms are proposed in this paper. For the big training data set, the method of " separate operation” is adopted to divide the data set into blocks. Each data block is inputted into Map nodes to calculate its error, and the errors of all data blocks are stored in the buffer. The block errors are read by Reduce nodes from the buffer to calculate the optimization objective function. Then, the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm is called to update the parameter set, and the updated parameter set is reloaded into the cluster. The above process is iterated until the optimization objective function converges; therefore, an optimal parameter set is obtained. For the big test data set, the parameter set obtained by the above steps is used to initialize the cluster. The vector representation of each sentence is calculated in Map nodes and temporarily stored in the buffer. Then, the sentiment label of each sentence is calculated by the classifier in the Reduce node using the vector representation. The experimental results demonstrate that in the standard MR (movie review) corpus, the accuracy of the algorithm is 77.0%, which is almost the same as the accuracy of the original algorithm (77.3%), at the same time the training time is decreased greatly along with the increase of compute nodes in the massive training data sets.
- semi-supervised recursive auto encoders,
- text sentiment analysis,
- parallel computing

FullText(HTML)

References(25)

References

LIU Bing. Sentiment analysis and opinion mining[M]. San Rafael: Morgan & Claypool, 2012: 1-6

唐慧丰,谭松波,程学旗. 基于监督学习的中文情感分类技术比较研究[J]. 中文信息学报,2007,6(2): 88-94.

TANG Huifeng, TAN Songbo, CHENG Xueqi. Research on sentiment classification of chinese reviews based on supervised machine learning techniques[J]. Journal of Chinese Information Processing, 2007, 6(2): 88-94.

梁军,柴玉梅,原慧斌,等. 基于深度学习的微博情感分析[J]. 中文信息学报,2014,28(5): 155-161. doi: 10.3969/j.issn.1003-0077.2014.05.019

LIANG Jun, CHAI Yumei, YUAN Huibin, et al. Deep learning for Chinese micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2014, 28(5): 155-161. doi: 10.3969/j.issn.1003-0077.2014.05.019

杨经,林世平. 基于SVM的文本词句情感分析[J]. 计算机应用与软件,2011,28(9): 225-228. doi: 10.3969/j.issn.1000-386X.2011.09.068

YANG Jing, LIN Shiping. Emotion analysis on text words and sentences based on SVM[J]. Computer Applications and Software, 2011, 28(9): 225-228. doi: 10.3969/j.issn.1000-386X.2011.09.068

HINTON G E, SALAKHUTDINOV R R. Reducing the dimensio-nality of data with neural networks[J]. Science, 2006, 28(7): 504-507.

HINTON G E, OSINDERO S A. Fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554. doi: 10.1162/neco.2006.18.7.1527

KIM Y. Convolutional neural networks for sentence classification[J]. Eprint Arxiv, 2014: 1746-1751.

黄磊,杜昌顺. 基于递归神经网络的文本分类研究[J]. 北京化工大学学报(自然科学版),2017,44(1): 99-104.

HUANG Lei, DU Changshun. Application of recurrent neural networks in text classification[J]. Journal of Beijing University of Chemical Technology (Natural Science), 2017, 44(1): 99-104.

SOCHER R, PENNINGTON J, HUANG E H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Edinburgh: John McIntyre Conference Centre, 2011: 151-161

SOCHER R, HUVAL B, MANNING C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island: Stanford Press Release, 2012: 1201-1211

SOCHER R, PERELYGIN A, WU J Y, et al. Recursive deep models for semantic compos-itionality over a sentiment treebank[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Seattle: Stanford Press Release, 2013: 1631-1642

DEAN J, CORRADO G S, MONGA R, et al. Large scale distributed deep networks[C]//Advances in Neural Information Processing. Vancouver: Curran Associates Inc., 2012: 1232-1240

RAINA R, MADHAVAN A, NG A. Large-scale deep unsupervised learning using graphics processors[C]//Proceeding 26th Annual International Conference on Machine Learning, ICML. Montreal: ACM. 2009: 873-880

温馨,罗侃,陈荣国. 基于Shark/Spark的分布式空间数据分析框架[J]. 地球信息科学学报,2015,17(4): 401-407.

WEN Xin, LUO Kai, CHEN Rongguo. Distributed spatial data analysis framework based on Shark/Spark[J]. Journal of Geo-information Science, 2015, 17(4): 401-407.

ABADI M, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning[C]// Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. Savannah: Google Press. 2016: 265-283

ISARD M, BUDIU M, YU Y, et al, Dryad: distributed data-parallel programs from sequential building blocks[C]//Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems. New York: ACM, 2007: 59-72

DEAN J, GHEMAWAT S. Mapreduce:simp-lified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1): 107-113. doi: 10.1145/1327452

侯佳林,王佳君,聂洪玉. 基于异常检测模型的异构环境下MapReduce性能优化[J]. 计算机应用,2015,35(9): 2476-2481.

HOU Jialin, WANG Jiajun, NIE Hongyu. MapReduce performance optimization based on anomaly detection model in heterogeneous cloud environment[J]. Journal of Computer Applications, 2015, 35(9): 2476-2481.

LIU Yang, YANG Jie, HUANG Yuang, et al. MapReduce based parallel neural networks in enabling large scale machine learning[J]. Computational Intelligence and Neuroscience, 2015, 2015(2): 1-13.

SUN Kairan, WEI Xu, JIA Gengtao, et al. Large-scale artificial neural network: mapreduce-based deep learning[DB/OL]. [2015-10-09]. https://arxiv.org/pdf/1-510.02709.pdf

ZHANG Kunlei, CHEN Xuewen. Large-scale deep belief nets with MapReduce[J]. IEEE Access, 2014, 2(2): 395-403.

PANG Bo, LEE L, VAITHYANATHAN S. Thumbs up? sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing. Philadelphia: Association for Computational Linguistics. 2002: 79-86

李然. 基于深度学习的短文本情感倾向性研究[D]. 北京: 北京理工大学, 2015

朱少杰. 基于深度学习的文本情感分类研究[D]. 哈尔滨: 哈尔滨工业大学, 2014

PANG Bo, LEE L. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceeding ACL’05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Michigan: Association for Computational Linguistics, 2005: 115-124

Relative Articles

[1]	LYU Gang, GUO Xilin. Calculation of Power Generation Characteristics of Linear Harmonic Generator for Electrodynamic Suspension Maglev Train[J]. Journal of Southwest Jiaotong University, 2023, 58(4): 783-791. doi: 10.3969/j.issn.0258-2724.20210892
[2]	HE Yanhui, GAN Yangjunjie, ZHOU Liang. Application of Active Magnetic Bearing in Waste Heat Generator[J]. Journal of Southwest Jiaotong University, 2022, 57(3): 657-664. doi: 10.3969/j.issn.0258-2724.20210860
[3]	CHEN Weirong, FU Wangxuan, HAN Ying, LI Qi, HUANG Lanjia, XU Chengpeng. Optimal Configuration of Wind-Solar-Hydrogen Multi-Energy Complementary Microgrid with Demand Side[J]. Journal of Southwest Jiaotong University, 2021, 56(3): 640-649. doi: 10.3969/j.issn.0258-2724.20200163
[4]	WANG Yingshu, WANG Xu, ZUO Yu, LIU Qing, ZHANG Juanjuan, YUAN Shu, YU Fucai. Review on Network Virtualization Simulation Software[J]. Journal of Southwest Jiaotong University, 2020, 55(1): 34-40. doi: 10.3969/j.issn.0258-2724.20190556
[5]	GAO Bo, HAO Changjin, LI Shuaibing, YIN Xiaobing, YU Xiaofeng, WU Guangning. Frequency-Domain Dielectric Spectroscopy Characteristics of Generator Epoxy/Mica Insulation[J]. Journal of Southwest Jiaotong University, 2017, 30(1): 9-15. doi: 10.3969/j.issn.0258-2724.2017.01.002
[6]	WANG Tianhong, LI Qi, CHENG Weirong. Adaptive Virtual Impedance Droop Control Method in Load Sharing[J]. Journal of Southwest Jiaotong University, 2017, 30(5): 1020-1028. doi: 10.3969/j.issn.0258-2724.2017.05.025
[7]	LI Donghui, ZHANG Jundong, JI Yulong. Stability Research for Parallel Operation of Diesel Generator Sets[J]. Journal of Southwest Jiaotong University, 2014, 27(3): 494-498. doi: 10.3969/j.issn.0258-2724.2014.03.019
[8]	HUANG Zhi-Wei, ZHOU Jian-Zhong, ZHANG Xiao-Yuan, WANG Chang-Jing, ZHANG Yong-Chuan. Dynamic Responses of Shaft System of Hydraulic Generator Set with Coupling Faults of Bearing Looseness and Rotor Rub-Impact[J]. Journal of Southwest Jiaotong University, 2011, 24(1): 121-126. doi: 10.3969/j.issn.0258-2724.2011.01.019
[9]	FU Xin-He, JU Ji-Bin, JI Wen-Juan. Principle Analysis and Implementation ofVoltage Control of Hybrid Excitation SynchronousGenerator[J]. Journal of Southwest Jiaotong University, 2010, 23(1): 99-104. doi: 10. 3969/.j issn. 0258-2724. 2
[10]	DING Guofu, HE Yong, ZOU Yisheng, WANG Jue, ZHANG Weihua. Virtual Prototyping-Based Feedback Model of Product Design Process[J]. Journal of Southwest Jiaotong University, 2008, 21(3): 367-372.
[11]	ZOU Yisheng, DING Guofu, ZHANG Weihua, LIU Jia. Design of Dispatching Platform for Development of Virtual Prototype of Rolling Stocks[J]. Journal of Southwest Jiaotong University, 2008, 21(1): 45-50.
[12]	LI Xueqin, PENG Qiyuan, FENG Wei, XIE Xiaosong. Model and PSO-Based Solution of Two-Dimensional Unbalanced Assignment Problem[J]. Journal of Southwest Jiaotong University, 2008, 21(4): 535-539.
[13]	HE Yong, DING Guofu, ZHAO Shujun, ZHANG Jihui, XU Mingheng. Object-Oriented Modeling and Manufacturing Simulation of Virtual Flexible Manufacturing Systems[J]. Journal of Southwest Jiaotong University, 2007, 20(5): 558-562.
[14]	JIA Diancun, WANG Bo. Design of Flexible Mechanism for Multi-Agent Based on Virtual Enterprises[J]. Journal of Southwest Jiaotong University, 2006, 19(6): 774-778.
[15]	GUO Lei, GAO Shibin, LI Qunzhan. Protection of Impedance-Matching Balance Traction Transformer Based on Transformer Model[J]. Journal of Southwest Jiaotong University, 2006, 19(6): 737-742.
[16]	LUTao. OperationalEquation and Setting ofDifferentialProtection for Impedance-M atching Balance Transformer[J]. Journal of Southwest Jiaotong University, 2005, 18(2): 158-162.
[17]	CHENXiao-shan, LAIXi-de, LIUXiao-bing. Development of Computer Simulation System for Overhauling of Hydroelectric Generating Unit[J]. Journal of Southwest Jiaotong University, 2004, 17(2): 234-238.
[18]	WULi-hong, ZHANGKun-lun, LUO Fan. Calculation of Induction Electromotive Force of Linear Generator of High-Speed Maglev Train[J]. Journal of Southwest Jiaotong University, 2004, 17(4): 460-464.
[19]	LI Gang-jun, CHEN Yong. Virtual Reality System for Robotics Application[J]. Journal of Southwest Jiaotong University, 2003, 16(1): 115-118.
[20]	ZHANG Xin-pei, CHEN Ying. An Improved Pseudo Variable Algorithm for Structural Reliability[J]. Journal of Southwest Jiaotong University, 2003, 16(1): 49-52.

Supplements(0)

Cited By

Cited by

Periodical cited type(4)

1.	李建林，梁策，曾飞，袁晓冬. 基于级联式模糊控制的电氢耦合直流微网能量管理策略研究. 电力系统保护与控制. 2024(18): 87-100 .
2.	李奇，邹雪俐，蒲雨辰，陈维荣，赵淑丹. 基于氢储能的热电联供型微电网优化调度方法. 西南交通大学学报. 2023(01): 9-21 . 本站查看
3.	高静兰. 基于复合储能的微电网平滑切换控制方法. 农村电气化. 2021(06): 57-61 .
4.	陈维荣，傅王璇，韩莹，李奇，黄兰佳，许程鹏. 计及需求侧的风-光-氢多能互补微电网优化配置. 西南交通大学学报. 2021(03): 640-649 . 本站查看

Other cited types(5)

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(9) / Tables(4)

Get Citation

PDF

XML

Article views(526) PDF downloads(20)