I.INTRODUCTION

According to the latest development plan for the railways system in China, in terms of railway network planning, China’s railway network will reach 150,000 km in 2020 and China’s railway network will reach 175,000 km in 2025. The scale of China’s railway construction has been expanding year by year, and its operating mileage has been increasing. Railway travel has also become the preferred mode of transportation for many passengers due to its comfortable, convenient, and cost-effective features. However, the accompanying safety issues have also caused great concern. Once an accident occurs, it will not only be extremely difficult to rescue, but also cause train delays, and even cause economic losses and casualties, which will seriously affect passenger travel.

During the travel of the railway, many factors will affect the safety of train operation, such as the incident of obstacles invading the railway line. Obstacles refer to all objects in the track area that are harmful to train operation, including pedestrians, debris flow, inspection and maintenance tools, windshields, etc. Once these obstacles intrude into the rail transit area, they are likely to cause train delays, and even cause serious economic losses or casualties. On February 10, 2017, a collision on a New York City subway train killed two people and seriously injured one. On December 25, 2018, Shanghai metro line 3 collided with a pedestrian who fell off a track, resulting in the death of a pedestrian. As the operation speed and traffic density of urban rail transit continue to increase, once the intrusion of obstacles occurs, it may cause huge losses to the life and property of the country and people.

At present, the main protection and monitoring methods for obstacle violations in urban rail transit are: in the case of manned trains, cameras are installed in critical areas or obstacles are judged by the driver looking ahead. For driverless trains, an obstacle detection system based on the principle of mechanical trigger is installed in the front end of the vehicle with a camera or the bottom end of the vehicle. However, in these two types of detection methods, the obstacle detection of the camera only displays the obstacle, and it cannot carry out real-time automatic detection and alarm. As for the contact obstacle detection method, the braking may not be timely or the vehicle body may be damaged, which is not conducive to the operation of the train and the safety of passengers.

Aiming at the above hazards in the operation of urban rail transit and defects in existing detection technologies, this paper adopts the noncontact multisensor technology detection method to detect the running environment of trains in real time and implement early warning. First, an obstacle detection device based on camera and lidar is installed beside the orbit to detect the obstacles in the orbit area in real time. Then, judge the obstacle information detected and determine whether the train should brake automatically or the driver should brake manually. Finally, this control command is transmitted to train control center and train driver through transponder, wireless base station, and other trackside equipment at different levels to realize train early warning and braking.

II.LITERATURE REVIEW

The most important thing in the detection of obstacles in rail transit is to divide the areas of interest related to the rail zone to be detected according to the collected video image information and ranging data, and then detect obstacles in the area of interest.

In the division of the region of interest (ROI), the most important step is the identification of orbital lines. Chinese and foreign scholars have conducted many studies and achieved certain results. Sungho Kim and others proposed an initial horizontal line detection and tracking method based on image and sensor, which can make the acquired images have a wider range of vision. Y. S. Tang and others proposed an improved Otsu threshold segmentation algorithm, which divides the gray levels into two categories according to image segmentation, thereby achieving lane line recognition. Kok Kiong Tan and others proposed a more powerful vision-based lane line detection method using a ridge detector and random sample consensus. Feng Jiao adopted a rail recognition algorithm based on Gabor edge detection operator and Hough transform, and proposed the application of a convolutional neural network algorithm to realize the recognition of rails and the division of dangerous areas. In railway operation, straight and curved tracks coexist. For different sections of railway lines, the detection methods of railway tracks are also different. After detecting the track line, the ROI is divided, and this region can be divided according to relevant safety regulations of the railway industry.

In terms of obstacle detection, the detection objects are diverse, and for static obstacles and dynamic obstacles, experts and scholars at home and abroad also have many research results. D. Sinha and others used accelerometer and Monte Carlo-based novel Bayesian analysis method to measure the vibration signal generated when an obstacle falls in the railway track, and this method could detect the instantaneous dynamic obstacle. J. J. Garcia and others used an infrared emission barrier and another receiver placed opposite the railway to establish different optical links to detect obstacles. When there is a dynamic or static obstacle, the link is interrupted. Qu Zhijian and others proposed a priority scheduling algorithm–kernel correlation filter (PSA-Kcf) algorithm for kernel-correlation filtering target tracking that integrates scale estimation, so as to achieve scale-adapted rapid tracking of foreign intrusions on railways. Wang Yao proposed a hardware design of railway foreign object detection algorithm based on field programmable gate array (FPGA), which implements image acquisition, moving object detection, single scan connected domain marking, and foreign object feature extraction. Obstacle detection can be divided into static obstacle detection and dynamic obstacle detection. The two detection methods can be fused by using specific detection methods. At present, a variety of noncontact sensor fusion methods can be used to detect obstacles and improve detection accuracy.

Therefore, this paper focuses on the research of obstacle recognition and detection methods, and then carries out subsequent information transmission and train control according to the detection results, so as to ensure the safety of train operation.

III.MODEL METHOD

This paper uses machine vision and lidar fusion technology to detect obstacles. It mainly detects obstacles in the ROI. When the obstacle reaches a certain height threshold, it is judged that the obstacle has an impact on the train operation. In the division of the region of interest, it is mainly based on the track line, and on this basis, it is divided according to the railway safety operation management regulations. In this technology, machine vision is mainly used to outline and shape the obstacles, and laser radar is mainly used to determine and locate the height information of obstacles. The integration of the two methods can detect the obstacles more accurately. It can control the train operation according to the hazard level of the obstacles by the automatic response mode or manual response mode of the signal system, so as to ensure the safety of the train operation.

A.TRACK-LINE DETECTION

The ROI is divided mainly to detect obstacles in the orbital zone and nearby dangerous areas. The division of the ROI region can reduce the detection range of obstacles and improve the detection accuracy. For the straight section and the curved section, different detection methods are adopted according to the different properties. The straight section uses the Hough transform based on the Sobel edge detection operator, and the curved section uses the Freeman chain code tracking algorithm based on Sobel operator.

1.HOUGH TRANSFORM BASED ON SOBEL EDGE DETECTION ALGORITHM

The edge detection algorithm is usually used to detect edge lines, and then the Hough transform is used to identify the track lines. Common edge detection operators are Roberts, Laplace, and Sobel.

Roberts operator is also called cross-differential algorithm. It is a gradient algorithm based on cross-difference, which detects edge lines through local difference calculation. It is often used to process steep low-noise images. When the edge of the image is close to +45° or −45°, this algorithm is more ideal. The disadvantage is that the positioning of the edges is not accurate, and the extracted edge lines are thick. Roberts operator templates are divided into horizontal and vertical directions:

dx=[−1001]dy=[0−110],
dx(i,j)=f(i+1,j+1)−f(i,j),
dy(i,j)=f(i,j+1)−f(i+1,j),
S=(dx(i,j)2+dy(i,j)2).

Laplace operator is a second-order differential operator in n-dimensional Euclidean space. It is often used in image enhancement and edge extraction. The Laplace operator is divided into four neighborhoods and eight neighborhoods. The four neighborhoods are gradients of the four directions of the center pixel of the neighborhood, and the eight neighborhoods are gradients of the eight directions. The four-neighbor and eight-neighbor templates are as follows:

H4=[0−10−14−10−10]H8=[−1−1−1−18−1−1−1−1].

Sobel operator is a kind of discrete differential operator for edge detection. It combines Gaussian smoothing and derivative. This operator is used to calculate the approximate value of the lightness and darkness of the image. According to the lightness and darkness of the edge of the image, a specific point in the area exceeding a certain number is recorded as an edge. Sobel operators have more accurate edge location and are often used for images with more noise and gray gradients. The algorithm template is shown in (6), where dx represents the horizontal direction, and dy represents the vertical direction:

dx=[10−120−210−1]dy=[−1−2−1000121].

Using python simulation, the three edge detection results are shown in Fig. 1.

Fig. 1. Comparison of edge detection algorithms.

Each picture in Fig. 1 represents background picture, Roberts test result map, Laplace test result map, and Sobel test result map. It can be seen from the comparison that Sobel has better detection effect on edge detection. Therefore, this paper uses the Hough transform based on Sobel edge detection algorithm to identify the track straight line segment.

The Hough transform has the advantages of extracting straight lines with high accuracy and strong anti-interference ability, and can traverse all edge pixels in the image and the straight lines that pass through the edge pixels in various directions. The principle of the Hough transform is for a straight line in a plane, in the Cartesian coordinate system, it can be represented by y = kx + b, where k is the slope and b is the intercept. However, if the straight line is a vertical line, k is infinite, so we usually represent the straight line in another coordinate system, r=x* cos (θ) + y* sin (θ) in polar coordinate system, (r, θ) can be used to represent a straight line, where r is the distance from the straight line to the origin, and θ is the angle between the vertical line of the straight line and the x axis.

For a given point (x0, y0), we draw all straight lines (r, θ) passing through it in polar coordinates, and we will get a sine curve. If all the sine curves of the nonzero points in the picture are drawn, there will be some intersection points. All sine curves passing through this intersection point indicate that they have the same (r, θ), which means that the points are on a straight line.

The three curves in Fig. 2 intersect at one point and have the same (r, θ). The Hough line transformation uses this method to find the straight line in the figure.

Fig. 2. Sinusoidal intersections.

2.FREEMAN CHAIN CODE TRACKING ALGORITHM BASED ON SOBEL OPERATOR

The track curve boundary extraction uses the Freeman chain code tracking algorithm based on the Sobel operator, which can be represented by an eight-direction Freeman chain code. The track curve is regarded as a collection of line segments with fixed length and direction. Select a pixel point on the image, and set eight pixels adjacent to it, and set them to 0∼7, as shown in Fig. 3.

Fig. 3. Freeman 8 direction value and pixel area.

Let i be the index value of a pixel on the image. If a curve is required to be pi→b7, the order of this curve can be pi→b4→b5→b6→b7, and the corresponding Freeman chain code is 46000. The resulting chain code is the digitized curve; the order of this curve segment may also be pi→b0→b7, and its corresponding Freeman chain code becomes 06.

Therefore, when using the Freeman chain code to lift the rail profile, the starting order must be specified, and then the binarization curve trajectory detected by the Sobel operator is extracted, and the extracted chain code is filtered according to the characteristics of the rail. The line segments are extracted, and the track boundary is obtained.

B.OBSTACLE DETECTION METHOD

1.COORDINATE SYSTEM FUSION

The trackside obstacle detection device mainly uses the fusion technology of machine vision and lidar. When the two are used together, the key is the fusion calibration and conversion of the coordinate system. The world coordinate system can be used as a conversion bridge to find the relationship between the two. The coordinate points (X′, Y′, Z′) of the lidar scanning point in the world coordinate system are

{X′=R cos ω sin αY′=R cos ω cos αZ′=R sin ω.
Among them, R represents the distance from the scanning point of the lidar to the center point; α represents the horizontal angle between the laser and the radar; ω represents the vertical angle of the emitted laser and the horizontal plane. The Y′ axis represents the direction pointed by the lidar, which is generally the direction of train advancement; the Z′ axis represents the vertical upward direction of the train; and the X′ axis represents the other direction that meets the right-hand rule.

The points (X′, Y′, Z′) in the world coordinate system can be expressed as:

[XYZ1]=[RT0T1][X′Y′Z′1].

Among them, R represents the rotation matrix relationship between the camera coordinate system and the world coordinate system; T represents the translation relationship between the two coordinate systems after rotation.

2.IMPROVED FRAME DIFFERENCE METHOD BASED ON PARTIAL BACKGROUND UPDATE

For obstacle detection in ROI region, this paper adopts improved frame difference method based on partial background update. The foreground distribution interval can be obtained from the foreground distribution sequence [ai, bi] (i ∈ (1, n)), where n is the number of intervals. For each distribution interval, the statistics of the foreground distribution along the other direction in this interval can be used to determine the rectangular position of each foreground concentrated distribution area. The set of these rectangular regions is defined as the background nonupdated region, which is described by the background mask (BM) (x, y):

M(x,y)={0,x∈[ai,bi] and y∈{y|Fi(y)=1}1,others,
where Fi (y) is the foreground distribution in the other direction in the interval [ai, bi]. BM (x, y) = 1 means that pixel (x, y) participates in background calculation, and BM (x, y) = 0 means that pixel (x, y) does not participate in background calculation.

After determining the area that is not involved in the background update, the current frame and the existing background are superimposed in a certain proportion to achieve the background update. The formula is as follows:

Bn(x,y)={Bn(x,y)=Bn−1(x,y)(x,y)∈{(x,y)|BMn(x,y)=0}(1−α)Bn−1(x,y)+αInBn−1(x,y)Bn−1(x,y)∈{Bn−1(x,y)|BMn(x,y)=1},
where Bn represents the updated background, and Bn−1 represents the existing background, α is the update rate, which satisfies α∈ (0,1). A larger value indicates that the current frame has a greater influence on the update background, that is, the faster the update speed. The above background update algorithm also requires an initial background, which is generally calculated by the following n frames:
Binit=∑i=1nIi(x,y)(x,y)∈{(x,y)|BMi(x,y)=1}∑i=1nBMi(x,y),
where Ii (x, y) is the ith frame image.

Using the background frame difference method, according to the information collected by the trackside obstacle detection device, the gray value of each frame image is subtracted from the background gray value to obtain a difference image id (x', y', i):

id(x′,y′,i)=f(x′,y′,i)−b(x,y,i0),
where b (x, y, i0)—the initial video image of the image acquisition device near the track inside the tunnel, that is, the background model; f (x', y', i)—each current frame image; (x, y)-2D image position coordinates; i—the number of image frames; i0—the number of initial image frames, generally 0.

By setting an image threshold T, a binary difference image can be obtained:

bid(x′,y′,i)={1,if  |id(x′,y′,i)|≥T0,if  |id(x′,y′,i)|<T.

According to the data collected by machine vision, the position and shape information of the obstacles are detected, supplemented by the height information and position information of the obstacle detected by the lidar, to determine whether the obstacle has an impact on the train operation, and judge the influence degree level, and then carry on the subsequent transmission of the test result information in grades.

C.OBSTACLE INFORMATION TRANSMISSION

The principle of the obstacle invasion limit alarm message disposal is to compress the time required for the alarm message to trigger the breaking of the first approaching dangerous train to the greatest extent possible. In order to minimize the impact of obstacle invasion limits on driving safety, and to reduce the rate of misjudgment, the detection results of obstacles are classified into danger levels. The classification of danger levels is judged by the location, size, height, etc., of obstacles. For obstacles with higher danger levels, the signal system direct response mode is adopted. After receiving alarm information, train control, radio block center (RBC) and other equipment approach the first. The dangerous train automatic train protection (ATP) onboard equipment sends a new driving permit (MA), or an unconditional emergency stop message. For obstacles with lower danger levels, the manual mode is used to transmit the obstacle information to the approaching trains, dispatching stations, station duty stations, etc., which are handled manually, and the trains are finally controlled. The obstacle information transmission diagram is shown in Fig. 4.

Fig. 4. Schematic diagram of obstacle information transmission.

IV.APPLICATION

According to the previous model method, python is used for simulation. The main simulation contents are identifying the trajectory line, dividing the ROI, and detecting obstacles.

A.TRACK-LINE DETECTION

According to the detected track line (Fig. 5), the range of Δu can be widened to the left and right according to the railway traffic safety criteria, which is the ROI. Dividing a straight section ROI into an example is shown in Fig. 6.

Fig. 5. Track-line detection.

Fig. 6. Region of interest (ROI) region diagram.

B.OBSTACLE DETECTION

According to the principle of partially updating the background, obstacle detection is realized, and the results are shown in Fig. 7. When there is an obstacle in the ROI area, according to the size and position of the obstacle, the height information of the obstacle detected by the lidar is added to determine whether it exceeds the set threshold range and detect it. In the experiment, the complex environmental conditions such as sunny day, cloudy day, and windy day were collected as the experimental basis, and the environmental influence was eliminated in the image preprocessing to the maximum extent, and the interference error of environmental factors was reduced as far as possible, so that the detection accuracy was higher.

In order to facilitate the analysis of the detection results, the horizontal projection and vertical projection of the detection results of frame 159 are shown in Figs. 8 and 9.

Fig. 7. Obstacle detection.

Fig. 8. Frame difference projection results.

Fig. 9. The projection results of the algorithm in this paper.

It can be seen from the above horizontal and vertical projection images that there are many noise points in the detection results of frame difference method, which makes it impossible to accurately locate foreign bodies. However, the improved algorithm in this paper can eliminate noise points, locate foreign bodies accurately, and achieve good detection effect.

C.OBSTACLE INFORMATION TRANSMISSION

When the obstacle information is detected in the ROI area, if the hazard level of the obstacle is high, the direct response mode of the signal system is adopted to transmit the obstacle information to the rail side equipment, and then it is transmitted to the train control or RBC through the global system for mobile communications-railway/general packet radio service (GSM-R/GPRS) network. And send a driving permission to the corresponding ground subsystem, the on-board subsystem, and the ground subsystem realize two-way communication, so as to control the first approaching train to achieve early warning and control. If the danger level of the obstacle is low, the manual processing mode is used to transmit the information to the driver or the dispatcher’s desk for manual judgment to control the train operation. Schematic diagram of obstacle information transmission is shown in Fig. 10.

Fig. 10. Information transmission diagram.

V.CONCLUSION

Obstacle detection technology of urban rail transit is an essential driving security technology in railway running system. Its detection method, detection accuracy, and processing of detection result information are very important. This paper mainly adopted the method of multisensor technology fusion to detect and judge the obstacles in the track area, and implemented the control mode of different response modes for the train according to the danger level, in order to guarantee the driving safety. The obstacle detection method described in this paper could realize the real-time detection of obstacles with high detection accuracy and good control mode, which can provide a safe and comfortable riding environment for urban rail transit and even railway traffic. Today, urban rail transit nationwide is focusing on creating smart travel, establishing key projects such as intelligent display systems, intelligent vehicle operation and maintenance platforms, and intelligent inspections. The technology developed in this paper focused on this direction, which significantly improved the automation and intelligence of the urban rail transit system and jointly guaranteed driving safety, which had great application value.