Utafiti huu unatengeneza mfumo jumuishi wa roboti unaoweza kutambua, kushika na kusafirisha zana za matengenezo zilizoko ardhini katika mazingira yasiyo sawa na yenye changamoto kubwa za kuona kama ballast ya reli. Mfumo unaunganisha vipengele vitatu vikuu: mtandao wa utambuzi wa vitu RA-YOLO unaotegemea YOLOv8n, vidole laini vya Fin Ray vyenye muundo usiolingana vilivyoboreshwa ili kuendana na zana nzito na zenye maumbo yasiyo ya kawaida, na usanifu wa udhibiti wa Simulink–ROS unaounganisha kamera ya kina Intel RealSense D435i na mkono wa roboti wa mhimili sita Aubo-i5.

Katika RA-YOLO, utaratibu wa attention wa CBAM na Mosaic data augmentation zilipotumika pamoja, thamani ya mAP@0.5 kwenye standard test set ilifikia %93,6. Model ilikuwa na parameters milioni 3,4 na ilifanya kazi kwa 105 FPS kwenye RTX 3090. Kwenye NVIDIA Jetson AGX Orin, kwa TensorRT na input resolution ya 640 × 640, ilifikia 42 FPS na source iliripoti end-to-end latency chini ya 25 ms. Katika additional ballast conditions, lowest mAP@0.5 ilikuwa %90,5 kwenye test iliyochanganya wet ballast na tofauti za texture/color.

Katika upande wa gripper, rib arrangement ilifanywa asymmetric ili kupunguza lateral instability ya standard symmetric Fin Ray structure inaposhika zana nzito. Large-deformation behavior ya TPU fingers ilifitiwa kwa third-order Yeoh hyperelastic model kwa kutumia experimental tensile data; goodness-of-fit coefficient iliripotiwa kuwa R²=0,9938. Baada ya finite-element na L9 orthogonal analysis, rib inclination ya 30°, rib spacing ya 8 mm na outer-wall thickness ya 3 mm zilichaguliwa.

Kwenye finger iliyotengenezwa physically, actuation displacement ilipoongezeka kutoka 1 mm hadi 6 mm, normal contact force iliongezeka kutoka 2,74 N hadi 15,47 N na experimental linear fingertip stiffness ikawekwa kuwa 2,56 N/mm. Baada ya 3000 repeated bending cycles, force ilishuka kutoka 15,47 N hadi 15,04 N, sawa na reduction ya %2,78. Pia, katika 500 lifting cycles zenye load ya 2 kg, hakuna visible slipping, load drop au visible structural damage iliyoripotiwa. Hizi ni indicators za short-term durability; long-term field fatigue haijatestwa katika study.

Katika system tests juu ya outdoor railway ballast, drill, hammer, pliers, screwdriver na wrench zilitumika. Katika isolated-tool condition, mean grasp success ilikuwa %95,2; katika harder scenario yenye partial overlap na stacked tools ilikuwa %88,4; na overall mean ya scenarios mbili ilikuwa %91,8. Hata hivyo, source haikueleza idadi ya independent grasp trials kwa kila tool na scenario, kwa hiyo sample size na confidence interval za percentages hizi haziwezi kutathminiwa.

Kwa mtazamo wa Uturuki, study inaweza kuwa technical example kwa tool collection katika railway-maintenance sites, assistant maintenance robots na robotic manipulation kwenye uneven terrain. Lakini experiments hazikufanywa kwenye railway infrastructure ya Uturuki, na heavy rain, snow, mud, dense dust, very low light, high-speed motion au long-term field use hazijavalidated. Kwa local application, ballast characteristics, lighting conditions, actual maintenance tools, safety procedures na robot platform vinahitaji field re-validation.

System inajaribu kutatua problem gani?

Katika railway-maintenance areas, tools hazipatikani kila mara kwenye flat na controlled factory floor. Ballast stones hutengeneza dense na irregular visual texture; metal tools zinaweza kuwa katika orientations tofauti, ku-overlap na kutoa strong reflections chini ya sunlight. Kwa hiyo system inahitaji si kutambua object pekee, bali pia kuamua position yake na ku-grasp irregular geometry yake reliably.

Approach ya study inagawanya problem hii katika layers tatu badala ya kutegemea single algorithm:

• Visual system inatambua maintenance tool.
• Depth na hand–eye calibration vinahamisha target kwenye robot coordinate system.
• Soft gripper inajirekebisha passively kwa object geometry na kutekeleza lifting operation.

Asymmetric Fin Ray finger ina tofauti gani?

Standard Fin Ray structure ni soft na adaptive lakini inaweza kuwa prone kwa buckling chini ya high lateral loads. Kwa kuwa researchers walilenga maintenance tools katika 2 kg class, finger geometry ilibadilishwa si kwa flexibility pekee bali pia kwa load-carrying capability.

Proposed structure ina three main geometric elements:

• Asymmetric inclined ribs: Zinasaidia finger kujikunja inward kuelekea object baada ya first contact.
• Curved outer wall: Inalenga kuongeza structural support wakati wa lateral bending.
• Silicone contact layer: Inaongeza friction dhidi ya slipping kwenye smooth na metallic surfaces.

Katika design requirements, total gripper weight ilitarajiwa kuwa 1 kg au chini, target object size 10–120 mm, na maximum load 2 kg au chini.

Large deformation ya TPU ilimodeliwaje?

Fingers zilitengenezwa kwa BASF Elastollan 1185A TPU kwa FDM method. Badala ya kukadiria material behavior directly kutoka catalog data, tensile specimens tatu za material ileile zilitestwa na mean stress–strain behavior ikatumika.

Energy density ya third-order Yeoh hyperelastic model ilifafanuliwa kama:

\[ W= \sum_{i=1}^{3} C_{i0}(I_1-3)^i \]

imefafanuliwa kwa muundo huu.

Kutoka experimental data katika 0–300% strain range, values hizi zilipatikana:

\[ C_{10}=2{,}054\ \mathrm{MPa} \]

\[ C_{20}=-0{,}0753\ \mathrm{MPa} \]

\[ C_{30}=0{,}00231\ \mathrm{MPa} \]

Goodness-of-fit coefficient iliripotiwa kuwa R²=0,9938.

Finger geometry ilioptimized vipi?

Katika finite-element analysis, three main variables zilichunguzwa: rib inclination angle \(\theta\), rib spacing \(d\), na outer-wall thickness \(t\).

Kisha L9(3³) orthogonal design ilitumika, na stress safety, adaptive deformation na lateral stability zikaunganishwa kuwa composite score moja:

\[ F=0{,}4f_{\sigma}+0{,}3f_{\delta}+0{,}3f_S \]

Stress safety ilipewa %40, adaptive deformation na lateral stability kila moja %30. Kati ya combinations tisa, configuration ya 30°–8 mm–3 mm ilitoa highest composite score ya 0,90.

Grasping force inategemea motor torque pekee?

Hapana. One important mechanical distinction katika study ni kuzingatia kwamba si energy yote inayotolewa na motor hubadilika kuwa clamping force kwenye object. Kwa sababu soft TPU finger inabending, sehemu ya energy huhifadhiwa katika elastic deformation.

Kwa hiyo source inachukulia finger kama flexible system yenye equivalent torsional stiffness kwa pseudo-rigid-body approach:

\[ M_{res}=K_{eq}\Delta\phi \]

na inaeleza contact force kama:

\[ F_c= \frac{1}{L_{eff}} \left[ \frac{\pi\eta L_1\sin\beta} {2P\cos\alpha}M_d - K_{eq}\Delta\phi \right] \]

inaelezwa kwa muundo huu.

Katika equation hii, motor torque inazalisha grasping force kupitia first term, wakati second term inawakilisha elastic counter-moment inayohitajika kwa TPU deformation. Physical interpretation ya source ni kwamba positive clamping force haiwezi kutokea kabla motor torque haijapita lower threshold fulani.

Ni condition gani inatumika dhidi ya slipping?

Lifting force inayozalishwa na fingers nne lazima ishinde gravity na motion acceleration. Theoretical stability condition imetolewa katika source kama:

\[ 4F_c(\sin\theta+\mu\cos\theta) \geq m(g+a_{max}) \]

imepewa kwa muundo huu.

Katika physical experimental validation, more conservative friction-dominated approach ilitumika:

\[ F_{available}=4\mu F_n \]

na required force:

\[ F_{required}=m(g+a_{max}) \]

ikahesabiwa. Maximum transfer acceleration ilichukuliwa kuwa \(1{,}0\ \mathrm{m/s^2}\).

Main change ya RA-YOLO ni nini?

RA-YOLO si large network inayobadilisha kabisa YOLOv8n architecture. Main improvement ni kuongeza Convolutional Block Attention Module (CBAM) kati ya C2f-P5 output ya Backbone na SPPF module, pamoja na kutumia Mosaic data augmentation katika training.

CBAM inatumia two sequential attention stages:

• channel-attention mechanism ina-weight feature channels kulingana na importance;
• spatial-attention mechanism inaamua ni image regions zipi zinahitaji focus.

Goal ni kusuppress features zinazojibu dense ballast-stone texture huku iki-strengthen features za metallic tool body.

Kwa nini Mosaic ilitumika?

Katika each training example, images nne zili-scale na crop kisha kuunganishwa kuwa single composite image. Hivyo scales tofauti, backgrounds na partial occlusions zinaweza kuonekana ndani ya same training sample. Figure 20 kwenye page 18 ya study inaonyesha Mosaic example ambapo image ya wrench moja imeunganishwa na tools tofauti na ballast fragments.

Matokeo yanayoungwa mkono na utafiti

• CBAM na Mosaic zilitumika pamoja na kuongeza mAP@0.5 katika research test set kutoka %85,4 kwa base YOLOv8n hadi %93,6.
• RA-YOLO yenye parameters milioni 3,4 ilifanya kazi kwa 105 FPS kwenye RTX 3090 na 42 FPS kwenye Jetson AGX Orin.
• Asymmetric Fin Ray structure ilitoa higher grasp success kuliko symmetric Fin Ray baseline katika all five tool classes katika controlled laboratory comparison.
• Contact force kwenye manufactured TPU fingers iliongezeka approximately linearly na actuation displacement.
• Baada ya 3000 bending cycles, reduction ya contact force iliripotiwa kuwa %2,78.
• Katika 500-cycle preliminary durability test yenye 2 kg load, hakuna visible load drop au slipping iliyoripotiwa.
• Overall grasp success katika outdoor ballast experiments ilikuwa %91,8.
• Overlapping-tool conditions zilionyesha lower grasp success kuliko isolated tools.

Matokeo ambayo utafiti hauungi mkono au bado haujatesti

• Haijaonyeshwa kwamba system inaweza kushika all railway maintenance tools; main experiment ilifanywa kwa five tool categories.
• Hakuna system-level validation iliyofanywa katika heavy rain, snow, mud, dense dust na extremely low light.
• 3000 finger cycles hazionyeshi long-term industrial TPU fatigue life.
• 500 grasping cycles za 2 kg hazithibitishi long-term reliability katika continuous field use.
• Hand–eye positioning result si direct measurement ya high-speed dynamic motion conditions; calibration validation ilifanywa semi-statically.
• Haijaonyeshwa kwamba %91,8 grasp success itabaki sawa kwa different robots, cameras, railway infrastructures na countries.
• Kwa sababu source haitoi total repetition count kwa success rates, statistical confidence intervals haziwezi kutathminiwa.
• Study haikutekeleza tactile-sensor closed-loop grasp-force control; hii imependekezwa kama future work.

Mbinu na Matokeo ya Utafiti

Visual dataset

Researchers waliunda total ya 3640 high-resolution images kutoka five maintenance-tool classes. Images zinajumuisha direct sunlight, cloudy weather na shadow conditions.

Separate environmental-robustness test set ambayo haikutumika katika training wala validation pia iliundwa. Additional test hii inajumuisha dry ballast, wet ballast, different ballast color/texture na wet + different ballast conditions.

RA-YOLO training

Ablation study ilionyesha nini?

Mosaic alone iliongeza mAP@0.5 kwa 2,4 percentage points relative kwa base model; CBAM alone ikaongeza 3,1 percentage points. Structure yenye both ilitoa mAP@0.5 higher kwa 8,2 percentage points kuliko base model.

Different attention mechanisms

Katika comparison hii, Mosaic ilitumika katika all models na attention module pekee ndiyo ilibadilishwa. CBAM ilitoa highest two accuracy metrics.

Algorithm-comparison error ndani ya source

Direct values za Table 11 ni hizi:

Baadhi ya percentage differences zilizotolewa katika paragraph inayofuata source hazilingani na table hii. Kwa hiyo raw mAP values katika table zinapaswa kuwa basis ya comparative interpretation.

Inaweza kufanya kazi kwenye Jetson AGX Orin?

Ndiyo. Study inatoa inference test kwenye real edge platform. RA-YOLO ilionyesha 105 FPS na approximately 9,5 ms latency upande wa RTX 3090 server, na 42 FPS pamoja na approximately 23,8 ms end-to-end latency kwenye Jetson AGX Orin kwa TensorRT.

Result hii ni hardware validation inayoonyesha model inaweza kufanya real-time image processing kwenye edge device; lakini haimaanishi entire robot-control loop inafanya kazi kwa 42 Hz.

Detection katika changing ballast conditions

Additional dataset hii ni separate kutoka main test set. Kwa hiyo especially mAP@0.5:0.95 values hazipaswi kulinganishwa directly na main test-table values kana kwamba ni same sample.

Hand–eye positioning

Kwa calibration, images za 10 × 7 checkerboard katika 15 different poses zilitumika. Katika subsequent validation, 50 random test points zilichaguliwa kutoka workspace.

Source inaripoti mean Euclidean positioning error ya 1,03 mm. Standard deviations za X na Y error distributions zimetolewa kuwa 0,32 mm na 0,28 mm mtawalia.

Hata hivyo, Figure 33(b) inaonyesha maximum Euclidean error ya 1,92 mm, wakati Table 13 ina example yenye single-axis absolute error inayofikia 2,4 mm. Source haielezi jinsi reporting forms hizi mbili zinavyopaswa kuunganishwa. Kwa hiyo hakuna single reconciled maximum-error value iliyotengenezwa.

Finger contact force

Experimental force–displacement relationship ilifitiwa kama:

\[ F_m=2{,}56\delta+0{,}11 \]

na linear fingertip stiffness ikawekwa kuwa:

\[ K_f=2{,}56\ \mathrm{N/mm} \]

Mean relative error kati ya model prediction na measured forces iliripotiwa kuwa %4,07.

Short-term cyclic durability

Test hii inahusu short-term cyclic stability. Source inaeleza wazi kwamba TPU fatigue katika long-term na high-frequency field use inahitaji kuchunguzwa separately.

Comparison ya asymmetric na symmetric Fin Ray

Katika controlled laboratory comparison, asymmetric design ilitoa higher success kwa all five tools. Lakini source haitoi number ya trials iliyofanywa kwa each success percentage.

Outdoor ballast grasp results

Figure 38 kwenye page 33 ya study inaonyesha physical setup ya robot si kwenye laboratory table, bali kwenye tracked mobile platform iliyowekwa katika ballast environment karibu na railway line. Aubo-i5 arm, D435i camera na asymmetric gripper zilitumika katika same system.

Highest overall success ilikuwa %94,7 kwa hammer, lowest ilikuwa %88,7 kwa wrench. Especially katika overlapping scenario, wrench success ilishuka hadi %83,6.

Waandishi wanahusisha hili na thin na flat wrench geometry kufanya effective multi-point contact na Fin Ray fingers kuwa difficult, na shiny metal surface kuweza kuathiri depth measurement negatively chini ya strong external light.

Task inachukua muda gani?

Mean task-completion time iliripotiwa kuwa 4,12 seconds katika isolated-tool scenario na 4,94 seconds katika overlapping scenario. Katika more complex scenario, time iliongezeka kwa sababu ya obstacle avoidance na environmental complexity.

Sababu za failures

Categories hizi nne zinajumlisha total failure rate ya %8,2. Source inaripoti kwamba detection-related misses zilionekana especially katika dense stacking cases ambapo occlusion ratio ilizidi %70.

Most important limitations

• Field tests zilifanywa katika representative outdoor conditions; all weather conditions hazikujumuishwa.
• Strong sunlight inaweza kusababisha point-cloud loss, especially katika active depth measurement.
• Main result ya positioning experiment ni semi-static calibration accuracy.
• Dynamic vibration, image latency na instantaneous depth fluctuation zilijaribiwa only indirectly katika system level.
• Gripper bado haina tactile/force-sensor closed-loop adjustment.
• Durability tests za 3000 na 500 cycles si long-term fatigue life.
• Repeat counts kwa field grasp-success percentages hazijaelezwa katika source.
• Results kutoka five tool classes haziwezi ku-generalize kwa all railway-maintenance equipment.

Maelezo ya Chanzo na Mbinu

Utafiti asilia: An Integrated Visual Perception and Soft Robotic Grasping System for Adaptive Handling of Railway Maintenance Tools.

Waandishi na order yao: Pan Fan; Meng Tian; Yuhang Du; Guodong Lang; Liang Li; Yafeng Li.

Corresponding author: Yafeng Li. Hakuna equal-first/equal-contribution statement katika source.

Taasisi: School of Computer, Baoji University of Arts and Sciences; Shaanxi Key Laboratory of Advanced Manufacturing and Evaluation of Robot Key Components, Baoji University of Arts and Sciences; State Key Laboratory of Robotics and System, Harbin Institute of Technology.

Jarida: Machines. Mchapishaji: MDPI. Volume: 14. Article: 636. Publication date: 1 June 2026. DOI: 10.3390/machines14060636. Official publication link: https://doi.org/10.3390/machines14060636

Source type na peer-review status: Ni peer-reviewed research article. Study ilipokelewa 8 April 2026, ikarevised 22 May 2026 na ikakubaliwa 27 May 2026.

Leseni: Creative Commons Attribution (CC BY) open-access license.

Funding: Ili-financewa na Shaanxi Provincial Department of Education Research Program chini ya project 23JP004.

Data availability: Source inaeleza kwamba data ziko ndani ya article.

Conflict of interest: Waandishi wametangaza kwamba hakuna conflict of interest.

Author contributions: Source inaeleza overall system design — Yafeng Li; final-draft review — Pan Fan; writing — Meng Tian; graphics — Yuhang Du; literature review — Guodong Lang; data analysis — Liang Li.

System architecture, RA-YOLO performance, TPU material model, FEA parameters, force experiments, cycle tests, hand–eye positioning results na grasp performance katika Verianla text hii zinategemea only study iliyochunguzwa. External bibliographic check ilitumika only kuthibitisha publication identity.

Kuna angalau two important reporting inconsistencies ndani ya source. Model accuracies katika Table 11 hazilingani mathematically na method-to-method difference values zilizotolewa katika paragraph inayofuata. Pia, ikiwa Figure 33's 1,92 mm maximum Euclidean-error presentation na Table 13's single-axis absolute error inayofikia 2,4 mm zinatoka kwenye same dataset, haziwezi zote kuwa sahihi kwa pamoja. Differences hizi hazijasahihishwa kimya kimya na zimehifadhiwa katika Verianla text kama source-internal uncertainty.

Ingawa study ina field evidence, validation imepunguzwa kwa representative, low-speed outdoor ballast conditions. Heavy rain, snow, mud, dense dust, extremely low light na long-term field wear hazijatestwa. Pia, kwa sababu independent trial counts kwa each system-level success rate hazijaelezwa, statistical confidence intervals haziwezi kuhesabiwa.

Abstract ya source inatumia term “visual servoing”. Hata hivyo, chain iliyoelezwa kwa detail katika methods section ni image detection, depth-based positioning, hand–eye transformation, inverse kinematics, polynomial trajectory planning na FSM control. Kwa sababu separate visual-servo control law inayotegemea continuous image-error feedback haijawasilishwa kwa detail, concept haijatafsiriwa kwa upana zaidi ya source katika Kiswahili.