用于云存储数据服务器的I/O请求调度算法

李宇

doi:10.3969/j.issn.0258-2724.2018.04.026

用于云存储数据服务器的I/O请求调度算法

doi: 10.3969/j.issn.0258-2724.2018.04.026

李宇

详细信息

作者简介:
李宇(1977-), 男, 副教授, 博士, 研究方向为软件工程、云计算及知识图谱, E-mail:liyu@bjtu.edu.cn

中图分类号: TP311.1
计量
- 文章访问数: 453
- HTML全文浏览量: 171
- PDF下载量: 77
- 被引次数: 26
出版历程
- 收稿日期: 2017-12-01
- 刊出日期: 2018-08-01

I/O Scheduling Algorithm for Data Servers in Cloud Storage Environments

LI Yu

摘要

摘要: 在云存储系统的体系架构中，当前对数据服务器守护进程的I/O请求调度采用先来先服务（first in first out）策略，这种调度策略没有考虑不同类型I/O请求的时效性要求，容易造成时效性要求高的I/O请求因被阻塞而无法得到及时处理，从而降低整个系统的服务质量.为解决该问题，本文提出一种用于云存储数据服务器的I/O请求调度算法.该算法首先对来自客户端的I/O请求进行分类，并赋予不同的优先级；然后以合适的时长作为周期、以分时间片的方式对不同优先级的I/O请求进行周期性的调度.分布式文件系统仿真实验结果表明，该算法在重负载情况下对实时请求的响应速度提高了20%，同时也兼顾了低优先级请求的响应性能.
- 大数据 /
- 云计算 /
- 分布式文件系统 /
- 周期性 /
- 优先级 /
- 调度算法
Abstract: Distributed file systems (DFSs) are generally employed for storing user data while designing a cloud storage system. The primary aspects of DFSs include efficient storage and management of metadata, data distribution strategies, and reliability of user data. In the case of data servers of a DFS, the FIFO (first in first out) strategy is adopted for scheduling I/O requests which are received by a data server daemon. The FIFO algorithm prioritises all such requests equally; requests that require better quality of services may therefore be blocked for long durations. To address this issue, a new priority based periodic scheduling algorithm (PPSA) has been proposed. Initially, PPSA classifies requests into different priority queues. Then, it periodically schedules requests according to their respective priorities and dedicated time slices. The obtained DFS simulation results show that PPSA can increase the response performance of heavy-load real-time requests by 20%, and can also ascertain the lowest response time performance for other requests.
- big data /
- cloud computing /
- distributed file system /
- periodic /
- priority /
- scheduling algorithms

HTML全文

图 1 PPSA工作原理

Figure 1. Working principle of PPSA

下载: 全尺寸图片幻灯片

图 2 周期性调度示意

Figure 2. Periodic scheduling process

下载: 全尺寸图片幻灯片

图 3 请求队列数据结构

Figure 3. Data structure of a request queue

下载: 全尺寸图片幻灯片

图 4 不同调度策略的系统响应时间比较

Figure 4. Comparison of system response times for different scheduling policies

下载: 全尺寸图片幻灯片

表 1 t₀类请求流数目对系统响应性能的影响

Table 1. Effect of the number of class t₀ request flows on the system response performance

t₀类请求流数量/个	FIFO			优先级调度			PPSA
t₀类请求流数量/个	t₀	t₁	t₂	t₀	t₁	t₂	t₀	t₁	t₂
20	2.92	2.96	2.95	1.82	2.39	3.82	1.83	2.42	3.82
40	2.97	2.97	2.98	1.90	2.72	4.22	1.92	2.73	4.25
60	2.95	2.95	2.99	2.02	3.12	4.63	2.05	3.12	4.61
80	2.93	2.96	2.91	2.20	3.62	4.94	2.15	3.47	4.70
100	2.99	2.97	2.93	2.31	4.02	∞	2.12	3.47	4.73

下载: 导出CSV

表 2 t₁类请求流数目对系统响应性能的影响

Table 2. Effect of the number of class t₁ request flows on the system response performance

t₁类请求流数量/个	FIFO			优先级调度			PPSA
t₁类请求流数量/个	t₀	t₁	t₂	t₀	t₁	t₂	t₀	t₁	t₂
50	2.66	2.66	2.65	1.99	2.68	3.32	2.02	2.72	3.28
100	2.96	2.98	2.97	2.17	3.09	4.10	2.12	3.00	3.97
150	3.02	2.99	2.98	2.19	3.36	4.54	2.15	3.21	4.51
200	2.98	2.98	2.99	2.17	3.57	4.93	2.13	3.42	4.76
250	2.97	2.96	2.98	2.18	3.73	∞	2.16	3.44	4.74

下载: 导出CSV

参考文献(18)

Zephoria Inc. The Top 20 valuable facebook statistics[EB/OL].(2018-04-25)[2018-06-05]. https://zephoria.com/top-15-valuable-facebook-statistics/

Dropbox Inc. Dropbox help center[EB/OL]. (2017-08-16)[2017-11-14]. https://www.dropbox.com/help.

Baidu Inc. Baidu mobile platform[EB/OL]. (2017-09-15)[2017-11-14]. https://pan.baidu.com/platform/home.

腾讯公司.微云会员存储空间构成[EB/OL]. (2017-06-13)[2017-11-14]. https://www.weiyun.com/vip/capacity.

Qihu360 Inc. 360 yunpan agreement[EB/OL]. (2017-03-15)[2017-11-14]. https://yunpan.360.cn/index/agreement.

XTAO P, HU Z G, LTU D B, et al. Energy-efficiency enhancedvirtual machine scheduling policy for mixed workloads in cloudenvironments[J]. Computers &. Electrical Engineering, 2014, 40(5):1650-1665. http://dl.acm.org/citation.cfm?id=2644083

王健宗, 谌炎俊, 谢长生.面向云存储的I/O资源效用优化调度算法研究[J].计算机研究与发展, 2013, 50(8):1657-1666. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jsjyjyfz201308009

WANG Jianzong, CHEN Yanjun, XIE Changsheng. Research on I/O resource scheduling algorithms for utility optimization towardscloud storage[J]. Journal of Computer Research and Developmen, 2013, 50(8):1657-1666. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jsjyjyfz201308009

郭松辉, 龚雪窑, 王炜, 等.一种动态优先级排序的虚拟机I/O调度算法[J].计算机科学, 2017, 44(1):13-19. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jsjkx201701003

GUO Songhui, GONG Xuerong, WANG Wei, et al. I/O Scheduling algorithm based on dynamic prioritization in virtual machines[J]. Computer Science, 2017, 44(1):13-19. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jsjkx201701003

WEIL S, BRANDT S, MILLER E, et al. Ceph: a scalable, high-performance distributed file system[C]//In Proceedings of the 7th Symposium on Operating Systems Design and Implementation.[S.l.]: USENIX Association, 2006: 307-320.

Red Hat, Inc. Gluster file system[EB/OL].(2017-01-21)[2017-06-05]. http://www.gluster.org/documentation/About_Gluster.

Apache Software Foundation. HDFS architecture[EB/OL]. (2017-09-15)[2017-11-14]. http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html.

LIU S, HUANG X, FU H, et al. Understanding data characteristics and access patterns in a cloud storage system[C]//In the 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, IEEE, 2013: 327-334.

KUROSE J, ROSS K. Computer networking:a top-down approach[M]. 6Ed. Hong Kong:Pearson Education Asia Limited, 2014:87-92.

AHMED A, DARRELL D E L, JEHAN F P, et al. File access prediction with adjustable accuracy[C]//In Proceedings of the 21st International Conference on Performance, Computing and Communications.[S.l.]: IEEE, 2002: 131-140.

ISHⅡ R P, MELLO R F D. An online data access prediction and optimization approach for distributed systems[J]. IEEE Transactions on Parallel and Distributed Systems, 2012, 23(6):1017-1029. doi: 10.1109/TPDS.2011.256

LIU Y, FIGUEIREDO R, CLAVIJO D, et al. Towards simulation of parallel file system scheduling algorithms with PFSsim[C/OL]//In Proceedings of the 7th IEEE International Workshop on Storage Network Architectures and Parallel I/O, 2011.[2017-08-12].http://storageconference.us/2011/Presentations.html.

VARGA A, HORNIG R. An overview of the OMNeT++ simulation environment[C]//In Proceedings of the 1st International Conference on Simulation Tools and Techniques for Communications, Networks and Systems & Workshops.[S.l.]: ICST, 2008: 60-70.

BUCY J S, SCHINDLER J, SCHLOSSER S W, et al. The disksim simulation environment version 4.0 reference manual (cmu-pdl-08-101)[R]. Pittsburgh: Carnegie Mellon University Parallel Data Laboratory, 2008.

施引文献

期刊类型引用(11)

1.	周福林，李琦琦，杨瑞轩，熊进飞，李群湛. V/v接线牵引变压器高次谐波横向传递特性. 中国电机工程学报. 2020(01): 290-299+395 . 百度学术
2.	陈民武，田航，宋雅琳，陈玲. 基于变频控制策略的同相供电装置可靠性优化方法. 西南交通大学学报. 2020(01): 9-17 . 本站查看
3.	骆冰祥，解绍锋，李群湛，苏鹏. 牵引网谐振特性与治理方案. 电网技术. 2020(05): 1957-1963 . 百度学术
4.	王广慧，张利军. 车网系统谐波模型及谐振特性仿真研究. 控制与信息技术. 2020(06): 42-49 . 百度学术
5.	赵文胜，罗杰，崔召华，黄军. 高速铁路AT供电系统谐波谐振频率研究. 铁道标准设计. 2019(08): 145-150 . 百度学术
6.	杨颢，陈民武，盛望群，刘巍，解绍锋. 新型贯通同相供电系统建模与运行特性分析. 铁道科学与工程学报. 2018(08): 2131-2139 . 百度学术
7.	谭智杰. 浅谈供电系统信息通信综合网控室的建立策略. 通讯世界. 2017(04): 74-75 . 百度学术
8.	袁琳. 高速铁路牵引变电所电源故障后的应急供电对策. 中小企业管理与科技(下旬刊). 2017(02): 173-174 . 百度学术
9.	乔沈楠. 高校供电系统的分析与节能的措施. 科学家. 2016(04): 79+81 . 百度学术
10.	赵元哲，李群湛，周福林. 基于阻波高通滤波器的高速铁路谐振抑制方案. 电力自动化设备. 2015(04): 139-144 . 百度学术
11.	赵元哲，李群湛，周福林，朱鹏. 牵引网高次谐波对高低压三相系统的渗透特性. 西南交通大学学报. 2015(05): 961-969 . 本站查看