The content of this article is from the Journal of Surveying and Mapping, Issue 1, 2024 (drawing review number: GS Jing (2024) No. 0107)
A multi-source fusion indoor positioning method with enhanced spatial relationship of "nearby".
Wang Yankun1, 2, 3
, FAN Hong4, FAN Yong3,5
, LI Xiaoming3, WANG Weixi3, GUO Renzhong3
1. Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources, Shenzhen 518034, Guangdong, China;
2. Institute of Internet of Things, Shenzhen Polytechnic, Shenzhen, Guangdong, 518055;
3. Institute of Smart City, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518061, Guangzhou, China;
4. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China;
5. School of Artificial Intelligence, Shenzhen Polytechnic, Shenzhen 518055, Guangdong, China
Funds: National Natural Science Foundation of China (42001389; 41971341); Guangdong Provincial University Key Area Project(2022ZDZX3071); The Key Laboratory of Urban Land Resources Monitoring and Simulation of the Ministry of Natural Resources of the People's Republic of China (KF-2022-07-024); Shenzhen Polytechnic Postdoctoral Outbound Late-stage Funding Project (6021271017K; 6023271011K); Shenzhen Polytechnic Project (6022312062K; 6023310002K)
Abstract:In view of the single traditional indoor positioning mode, combined with the "nearby" spatial relationship commonly used in indoor location description, and the fusion of multi-sensor data, this paper proposes a multi-source fusion voice interaction indoor positioning method with enhanced "nearby" spatial relationship. Firstly, the characteristics of the spatial relationship between "nearby" are studied, and the probability density function of the spatial relationship between "nearby" based on "stealing area" and the shortest distance is established for the indoor environment. Secondly, the fingerprint information of each reference node and the distance and motion information between the nodes were collected, and the indoor location description and positioning process was modeled based on the hidden Markov model, and the user position was predicted by the Vibit algorithm. Finally, the method is verified by actual scenarios, and the average positioning accuracy of the proposed method can reach 2.12 m when the average positioning accuracy is 1.88 m, and 80%.
Key words: spatial relationship of "nearby", multi-source data fusion, indoor positioning, voice interaction
WANG Yankun, FAN Hong, FAN Yong, et al. A Multi-source Fusion Indoor Localization Method with Enhanced "Nearby" Spatial Relationship[J]. Journal of Surveying and Mapping,2024,53(1):118-125. DOI: 10.11947/j.AGCS.2024.20230019
WANG Yankun, FAN Hong, FAN Yong, et al. A "near" relation enhanced multi-sourced data fusion indoor positioning method[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(1): 118-125. DOI: 10.11947/j.AGCS.2024.20230019
Read more: http://xb.chinasmp.com/article/2024/1001-1595/20240110.htm
INTRODUCTION
Indoor positioning is a key technology for many Internet of Things (IoT) applications, such as transportation, emergency response, and medical care [1-2]. According to statistics, the global indoor positioning market is expected to reach $17 billion by 2025. At present, both academia and industry are sparing no effort to promote indoor positioning technology to meet the needs of different scenarios [3].
At present, indoor positioning technology is in full bloom. Indoor positioning methods can be divided into single-source positioning and multi-source fusion positioning methods. Single-source positioning methods include Wi-Fi, pedestrian dead reckoning (PDR), Bluetooth Low Energy (BLE), and ultra-wideband (UWB) [4]. Wi-Fi positioning methods have attracted extensive attention due to the popularization of infrastructure, but the construction of fingerprint database requires a lot of labor and time costs [5], and pedestrian track estimation does not rely on infrastructure to achieve positioning, but requires an initial location [6]. Due to the complex and dynamic changes of the indoor environment, the single-source positioning method based on the above is not effective in practical application scenarios. Therefore, the multi-source fusion localization method with complementary advantages has attracted the attention of many scholars. Multi-source data fusion schemes can be divided into loosely coupled and tightly coupled [7]. Among them, the loose coupling is based on the fusion of the positioning results of different sensors, which is easy to achieve, but due to the heterogeneity (non-homologous) of various sensors, the coefficient of the output positioning results entering the information fusion center is not easy to be obtained by analytical methods. The close coupling is based on the observation and measurement of different sensors, such as Kalman filter, extended Kalman filter, etc., the method obtains the position and heading of the pedestrian through the observation values (speed, heading angle, step size, etc.) of each sensor, and determines the observation equation and the equation of state to calculate the position of the pedestrian. The performance of multi-source fusion positioning is better than that of single-source positioning, but it will also lead to an increase in positioning cost and terminal energy consumption [8]. With the continuous enhancement of the ability of mobile phone sensors to perceive the surrounding environment and user behavior, the method of improving positioning accuracy or stability through semantic perception has attracted more and more attention [9]. With the excavation of users' spatial cognitive ability by smartphones, indoor positioning technology is gradually developing from perception to cognition [10]. In the era of artificial intelligence, users' demand for indoor positioning is not only for accuracy, but more importantly, for better integration with intelligent terminal devices and intelligent services [11]. Therefore, it is of great practical significance to develop an indoor positioning method that takes into account positioning accuracy, intelligence and low cost combined with spatial cognition.
As the most natural and convenient way of communication in the construction of smart cities, voice interaction will become the most important interaction mode for smart devices and products. The development of natural language processing technology has made the interaction between humans and machines more mature [12]. As a form of speech, location descriptions are widely used in people's daily communication, especially in the process of navigation and positioning, such as "I am near McDonald's". Location description is the user's description of spatial cognition, which is a complex set with characteristics such as ambiguity, hierarchical expression, and contextual environment [13-14]. The complete location description includes the reference object, spatial relationships, and target objects [15]. The spatial relationship in the location description is an external expression after people generalize, abstract, and summarize the relationship between real space objects [16-17]. In this regard, the spatial relationship in the location description can be divided into distance relationship (qualitative distance and quantitative distance), orientation relationship (relative and absolute orientation), and topological relationship [18]. Compared with the topological relationship, the distance relationship and the azimuth relationship convey more position cues, which are suitable for positioning. Commonly used location description localization methods include the point method, the point-radius method, the probability density function (PDF), and the shape method, among which the PDF method considers the shape of the reference object [19]. Ref. [20] discussed the uncertainty of the user's description of location indoors with quantitative distance and orientation based on joint probability. However, as a commonly used spatial relationship in location description, "nearby" can determine the user's approximate location, but it is affected by cognition and has uncertainty and cannot meet the positioning needs [20]. Wi-Fi and PDR localization methods have attracted much attention without laying additional infrastructure, and they can enhance the cognitive space area of the location description and reduce its uncertainty. To this end, this paper proposes a multi-source fusion speech interaction indoor localization method based on "nearby" spatial relationship enhancement.
1. Modeling and positioning of spatial relationships
Figure 1 shows the flow of the research method proposed in this paper, and the data input of the method includes real-time RSSI fingerprints, reference objects and their spatial relationships, and built-in sensor data of mobile phones. (1) The RSSI fingerprint obtained in real time is matched with the fingerprint database, (2) the "nearby" spatial relationship function of the reference object is constructed based on the fuzzy set, and (3) the status information (step size, cadence, etc.) of the pedestrian is obtained based on the built-in touch sensor of the mobile phone. Based on the above observation data, a hidden Markov model is constructed, and the position of pedestrians is estimated by the Vibit algorithm.| 图 1 本文方法流程Fig. 1 The flowchart of the proposed method |
| Diagram options |
1.1 Characteristics and modeling of "nearby" spatial relationships
1.1.1 "Nearby" spatial relationship characteristics
The spatial relationship of "nearby" belongs to quantitative distance, which is an abstract expression of people's cognition of distance relationship, which has the characteristics of ambiguity and multi-scale.
(1) Feature 1: Ambiguity. From the perspective of cognitive linguistics, the spatial relationship of "nearby" is the external qualitative expression of people's perception of spatial distance, that is, the process of completing the transformation of reality → cognition → language. Factors influencing distance perception include age, background, and environment [21-24].
(2) Feature 2: Multi-scale. Different users have different application and analysis requirements for geographic data, resulting in different representation problems called multi-scale problems. The spatial relationship of "nearby" has different manifestations at different scales, which is manifested in the cognitive difference of distance. For example, the understanding of the relationship between "nearby" in the campus [25] and the understanding of the concept of "nearby" at the urban scale in the state of driving [26-27].
1.1.2 Modeling of "nearby" spatial relationships
For the concept of "nearby", there is a general consensus on spatial cognition, that is, the closer to the reference object, the greater the membership degree of "nearby". At present, qualitative distance modeling methods mainly include cognitive experiments, geographic information retrieval (GIR) and geometric construction. Cognitive experiments are time-consuming, labor-intensive, and not practical, and geographic information retrieval methods require the help of social media or other spatial data that indirectly reflect people's cognition, which is suitable for small-scale spaces and is not suitable for indoor scenes. In this paper, we refer to the method in Ref. [28] to establish a Voronoi diagram of polygon features, and model the spatial relationship between the "nearby" based on the "stolen area" and Euclidean distance of the Voronoi diagram. The modeling process is as follows.
Step 1: Build a proximity area. As shown in Figure 2(a), the Voronoi region of the Voronoi region of the features RO1, RO2, RO3, RO4, RO5, RO6, RO7, RO8, RO9} is constructed with the reference object set {RO1, RO2, RO3, RO4, i.e., neigh(RO1)={RO2, RO3}. The vertices of the Voronoi polygon region of RO1 are v1 and v2, where v2 is the common vertex of RO2, RO3 and RO1, and the closest vertices to vertex v2 on RO2 and RO3 are s1 and s4, and the circumscribed circles of v2, s1 and s4 triangles are drawn to obtain the arc between RO2 and RO3
in the same way as an arc segment between other reference objects, which are connected by the edges of the reference object (e.g., the line segment s1s2 belonging to RO2 connects the arc with
to form a polygon surrounded by arcs and straight lines, i.e., NeighArea (RO1).
| |
| 图 2 窃取面积形成过程Fig. 2 The construction process of stolen area |
| Diagram options |
Step 2: Generate the stolen area. When the new growth point t is inserted into the existing Voronoi diagram, the Voronoi diagram is reconstructed to cause the area of the other reference objects Voronoi to be reduced. As shown in Figure 2(b), the area enclosed by the blue line is the area stolen from the reference object RO1, RO2, RO3, and RO6 after the new growth point t is inserted into the existing Voronoi diagram, and the shaded area is the area stolen from the reference object RO1.
Based on the Euclidean distance and the stealing area, the probability function of the "nearby" spatial relationship is established
(1)
where t represents the new growth point, t∈NeighArea(Ri), min d(t, R) is the shortest distance from t to the reference object Ri, and Ak is the area that t steals from the reference object. Figure 3 shows the probability distribution of the spatial relationship "nearby" of RO1 according to Eq. (1).
| |
| 图 3 RO1“附近”关系的概率分布Fig. 3 Near relationship probability distribution of RO1 |
| Diagram options |
1.2 Fusion Positioning Algorithm
1.2.1 Construction of hidden Markov mathematical models
The pedestrian localization process can be expressed using the hidden Markov model {H, O, A, B, π}, as shown in Figure 4.
| |
| 图 4 隐马尔可夫模型Fig. 4 Hidden Markov model |
| Diagram options |
H represents the hidden state S={H1, H2, ..., Ht, }, Ht represents the hidden state at time t, and H represents the anchor point in this system.
O represents the observation state O={O1, O2, ..., Ot}, Ot represents the observation state at time t, and the observation state in this system includes the received RSS fingerprint signal and motion information.
A represents the transition probability A={aij}, that is, the probability that Hi will be transferred to the next anchor Hj, aij=P(Hj|). Hi)。
B represents the emission probability B={bj(t)}, and each hidden state corresponds to an observation state, that is, the probability of receiving a signal from a reference anchor, bj(t)=P(Ot|). Ht)。
π represents the initial probability distribution, and the probability distribution of the user's position at the beginning of the positioning is the same.
The core steps of the above process are the determination of the transfer probability and the launch probability.
1.2.2 Probability of Transfer
The state transition probability of the system consists of two parts, namely, the distance observation error and the angle observation error distribution. The formula for the distance and angle transition probability from state Hi to state Hj is
(2)
(3)
where Hi and Hj are the hidden states of nodes i and j, k is the displacement, dij is the distance between nodes, σk is the average error of the measured displacement, θ is the angle, φij is the heading angle, and σθ is the average error of the measurement angle.
Since distance and angle observations are relatively independent, for this reason, the state transition probability can be expressed as
(4)
1.2.3 Probability of launch
The transmission probability of the system consists of two parts, namely the RSS fingerprint and the "nearby" spatial relationship
(5)
In the formula, RSS is the fingerprint information measured in real time, that is, RSS=(rss1, rss2, ..., rssq), σix is the signal strength deviation at node i, q is the number of APs, fix and rx are reference fingerprints and real-time fingerprints, respectively, and x is the AP number.
The probability of emission for the "nearby" spatial relation is Eq. (1), i.e
(6)
The RSS fingerprint is independent of the "nearby" spatial relationship, and the emission probability can be expressed as
(7)
When there is no "nearby" spatial relationship in the input, the transmission probability is the RSS fingerprint probability obtained in real time.
1.2.4 Online Positioning Process
The online positioning process is implemented by the Vibit algorithm. As the number of reference points increases, the computation time increases. To this end, this paper selects the reference points through the "nearby" region to form a set of candidate reference points to improve the calculation efficiency. Figure 5 shows the candidate reference points generated and filtered based on the RO1 "nearby" region.
(8)
| |
| 图 5 RO1“附近”区域内参考点Fig. 5 The candidate sets of NeighArea (RO1) |
| Diagram options |
where δt(j) is the forward probability, δt-1(i) is the probability of the node at the previous time, and P(A) and P(B) are the state transition probabilities and emission probabilities, respectively.
When t=1, the forward probability is the product of the initial probability and the launch probability
(9)
When t>1, the forward probability at time t is the product of the forward probability, the launch probability and the transfer probability at time t-1, and the highest probability is the anchor point
(10)
2. Test verification and analysis
In this paper, the voice interaction module uses the iFLYTEK platform software to complete the speech-to-text conversion based on the speech conversion module, and extracts the "nearby" spatial relationship and related geographic entities in the text location description through word segmentation technology.
The test site was selected on the first floor of a large commercial plaza in Shenzhen, and a high-precision two-dimensional vector map was obtained through indoor three-dimensional point cloud data, and the floor plan is shown in Figure 6, in which the yellow, red and blue circles are the "nearby" areas of L2-18, L2-48 and L2-50 respectively. Considering the characteristics of the indoor structure, Wi-Fi fingerprints were collected every 1.5 m along the store. 5 participants were invited to participate in the trial, and the age composition of the participants was 18~52 years old, including students, civil servants, retailers, etc. Participants were asked to walk indoors randomly with a positioning terminal in hand, and describe their position through the spatial relationship of "nearby" at any position. Voice location description data and sensor data (Wi-Fi, magnetometer, accelerometer, etc.) are recorded in the background. In order to ensure that the data of each group of participants are comparable, each participant walked indoors for 2~3 laps, and the number of collected data was basically the same.
| |
| 图 6 室内平面Fig. 6 The indoor plan |
| Diagram options |
According to the classification statistics of positioning accuracy according to the participants, Figure 7 is the cumulative function curve of positioning probability, and Table 1 is the statistics of correlation error. The minimum positioning error of participant 1 (P1) was 0.21 m, the maximum error was 2.80 m, and the average error was 1.79 m, the minimum positioning error of participant 2 (P2) was 0.09 m, the maximum error was 2.69 m, and the average error was 1.69 m, the minimum positioning error of participant 3 (P3) was 0.34 m, the maximum error was 2.64 m, and the average error was 1.54 m, and the minimum positioning error and the maximum error of participant 4 (P4) was 0.09 and 2.88 m, the average error was 1.79 m, and the minimum positioning error of participant 5 (P5) was 0.01 m, the maximum error was 2.92 m, and the average error was 1.66 m. Statistics show that there is little difference in the positioning results of each participant. To a certain extent, it is proved that the positioning accuracy of the method in this paper is not much different from that of individual participants.
| |
| 图 7 误差累计函数Fig. 7 The cumulative distribution functions (CDFs) of different participants |
| Diagram options |
表 1 不同被使者误差Tab. 1 Positioning error statistics of different participants
In order to verify the effect of the "nearby" spatial relationship in the fusion localization method, the experimental scheme in this paper is divided according to whether the "nearby" spatial relationship is considered, that is, the "nearby" spatial relationship is fused with Wi-Fi/PDR (Near+Wi-Fi + PDR), the fusion Wi-Fi and PDR (Wi-Fi + PDR), and Wi-Fi positioning. The cumulative error probability function is shown in Figure 8, and the related error statistics are shown in Table 2. The average error of the Near+Wi-Fi+PDR method is 1.88 m, the minimum positioning error is 0.25 m, and the maximum positioning error is 2.8 m, the average error of the WiFi+PDR positioning method is 2.69 m, the minimum positioning error is 0.38 m, and the maximum positioning error is 5.8 m, and the average error of the Wi-Fi positioning method is 3.45 m, the minimum positioning error is 0.18 m, and the maximum positioning error is 5.9 m. Near+Wi-Fi+PDR performed better than Wi-Fi+PDR and Wi-Fi overall, and the WiFi location results of a single location source performed the worst. The maximum error of WiFi+PDR and Wi-Fi positioning methods is 5.8 m due to the presence of fingerprint blur in the experimental scene, and the addition of "nearby" spatial relationship can constrain the fingerprint spatially, reduce the fingerprint ambiguity, and improve the matching efficiency.
| |
| 图 8 不同定位方法精度对比Fig. 8 The cumulative distribution functions (CDFs) of different methods |
| Diagram options |
表 2 不同定位方法误差统计Tab. 2 Positioning error statistics of different methods
3 Conclusion
With the development of indoor positioning technology, people's demand for indoor positioning is not limited to the improvement of accuracy, but also pays more attention to the integration with intelligent terminals. In view of the single traditional indoor positioning mode, combined with the commonly used "nearby" spatial relationship in location description, and the fusion of other multi-source sensor data, an exploratory speech interactive indoor localization method based on hidden Markov model was proposed, and the effectiveness of the proposed method was verified by experiments. Based on the comparative analysis of different subjects, there is little difference in the cognitive perception of the participants, and the experimental comparison shows that the "nearby" spatial relationship in the fusion localization method can reduce the ambiguity of fingerprints, improve the efficiency of fingerprint matching, and then improve the positioning accuracy, with an average positioning accuracy of 1.88 m. The method in this paper is used for human-computer interaction voice navigation and positioning, which can provide a reference for the navigation and positioning of blind people.
However, the method in this paper only integrates the "nearby" spatial relationship, and the position description also includes other spatial relationships that can assist in positioning, such as quantitative distance, azimuth relationship, and topological relationship. These will continue to be deepened in future research work.
About the Author
About the first author:WANG Yankun (1988-), male, Ph.D., distinguished associate professor, research direction is indoor navigation and positioning, spatial cognition. E-mail: [email protected] Corresponding author: Fan Yong E-mail:[email protected]
First trial: Zhang Lin review: Song Qifan
Final Judge: Jin Jun
information
Wu Hongtao, Information Center of the Ministry of Natural Resources: Concept Evolution, Connotation Analysis and Overall Architecture of Smart Land: A Case Study of Chongqing Municipality | Journal of Spatio-temporal Information Technology, No. 1, 2024