Espacenet- Bibliographic data
`
`Page | of 1
`
` Espacenet
`
`Bibliographic data: JP2O71S5733078 (A} — 2018-07-23
`
`OBJECT RECOGNITION DEVICE FOR CONTROL, MOBILE APPARATUS
`CONTROL SYSTEM, AND PROGRAM FOR OBJECT RECOGNITION FOR
`CONTROL.
`
`Inventor(s):
`
`Applicant(s):
`
`Classification:
`
`Application number:
`
`GUANHAIKE + (68 3853)
`
`RICOH CO LTD + (#kxlestt Y 4 —)
`
`-international:
`- cooperative:
`JP20140005417 20140115
`
`G067T7/00; GO6T?7/20; GO8G1/16
`
`Global Dossier
`
`Priority number(s):
`
`JP20140005417 20140115
`
`Also published as:
`
`JP6315308 (B2)
`
`Abstract of JP2078133078 (4}
`
`PROBLEM TO BE SOLVED: To achieve the appropriate control of predetermined
`equioment mounied on a mobile body by appropriately identifying whether or not an
`object existing in the periphery of the mobile body is an obiect for controh SOLUTION:
`An abject recognition part 205 detects a detection ablect such as another vehicie ora
`pedestrian existing in ihe perighery of a mobile body on the hasis of a pickup image
`obtained by picking up the periphery of the mobile body. On the other hand, an object
`state determination part 207 specifies relative positions Dz, X of the detection abject
`detected by the cbiect recognition part to the mobile body on the basis of parallax
`information acauired fram 4 pair of pickup images picked by a stereocamera, and
`apeciies relative speeds Vx, Vy, Vz of the detection oblect to the mobile body from a
`difference between ihe image positions of the detection object between two or more
`pickup images (frames) whose pickup periods are different, An object recognition part
`208 for control determines whether or not the specified relative positions and relative
`speeds satisfy the conditions of the relative positions and relative speeds as an cbiect
`for cortral,
`
`https://worldwide.espacenet.com/publicationDetails/biblio?J=1&ND=3 &adjacent=true&loc... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 1 of 13
`
`Powered by EPO and Google § s
`~~«te,
`
`rg
`
`Patent Transiate
`
`nerated. I cannot be guaranieed thai itis intelligible, accurate, complete, retiable or ft for specific purposes, Critical decisions,
`This translation is machine-ge!
`such as commercially relevant or financial decisions, should not be based on machine-transiation oufsut,
`
`
`
`
`
`WwW Le GaAvg
`GRE TOTISR FS
`
`An oblect of the present invention is ia appropriately identify whether or qot an cbhiecl prasent around a moving object is a
`contral object, and ta enable apprapriate cantrol of a predetermined device mounted on tne moving oblect. An oblect recognition
`unt (205) detects 3 detection target such as another vehicle ar a pedestrian present around a moving body Gasecd on a captured
`
`
`image obtained by Imaging the perghery of the moving body. On the ather hand,
`the oblect slate determination unit 207
`specifies the lative position Dz, X of the detection target detected by the object recognition unt with respect to the moving
`
`obiect based on parallax information obtained from a pair of captured images captured by a stereo camera. The relaiive
`
`velocities Vx, Vy, Vz of the detection langerelative to the moving objact are specified frorn the differance in ihe image pasiien
`of the detection target hetween Iwo or more cantured images
`(between frames} having diferent imaging timings. The contra!
`
`object recognition unit 238 determines whether or not the specified relative position and relative velocity satisfy the conditions of
`ihe rehative pasitien and the relative velocity fo be the contral oblect. [Selected figure} Figure 4
`
`Contral object recognition device, mobile device contral system and cuntrol object recognition program
`
`fond)
`
`The present invention is provided with a control target identification device for identiying whether or not a detection target
`present around a mobile is a contral target for controlling a predetermined device mounted on the mobile, The present inventien
`relates to a mobile device control system and a contro! abject recognition program.
`
`(8002)
`
`Conventionally, as & device for detecting an oblect exisiing around a moving bady, for exam@le, a driver suppart system (mobile
`bady such as ACC (Adaptive Cruise Control for reducing a driving load of a driver of a vehicle (mobile body} What is used for
`
`ment contral systems etc. is Known.
`
`
`The driver assistanoe system cantrois varous devices mauntacd on the host vehicle, andi has an autornatic brake funcilon and an
`alarm for preventing the host vehicle from colliding with an obstacie or the like and reducing the impact in the event of a
`collision.
`it realizes various functions such as function, sel-vehicie speed adjustment function to maintain inter-vehicie distance
`with leading vehicis, and functian fo sucpart prevention of depariure fram ihe tane where both vehicies are traveling.
`
`i003)
`
`in Patent Document 7, distance data (parallax value) is calculated from a pair of captured images obtained by capturing an area
`in frant of the hast vehicie by a sierea carmera mounted an the hast vehicle, and above the road surface based on the distance
`data Discluses = three-dimensional object detection apparatus for detecting a three-dimensional object present in in this three-
`dimensional oblect detection device, the three-dimensional object detected in the processing target Image (one of the pair of
`captured images} captured af a certain time is regarded as a stationary obiect, and the motion state of the steres camera Using
`ui
`
`the detection result
`of the vehicle speed information and the distance data of the three-dimensional obiect,
`the image
`
`coordinates at which the three-dimensional oblect aonears at the next dme are oredicted. When the three-dimensional abject is
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 2 of 13
`
`a stationary object, the diference between the luminance value of the image area of the three-dimensional obiect and the
`luminance value of the image coordinates related io ihe grediction of the image actually captured at the next time is zero. When
`the three-dimensional oblect is a moving abiect, the difference is not zero. Thus, according fo Patent Document 1, 1 is passibie
`to detect a moving obiect moving in the front region of the host vehicle at a high processing speed.
`
`i004}
`
`is necessary fo contra!
`ft
`in order to properly function a mobile device control sysiem such as a driver assistance system,
`predetermined devices on the mobile obiect controlled by the mobile device contre] system by various objects shown in the
`captured image. itis imporiant to accurately identify whether it is a control object or not. Han obiect that is a non-contral ohiect
`is recognized as a control object,
`if causes a malfunction that performs device contra! thal is not necessary originally, and
`conversely, an obiect that is a contro! object is not if itis recognized that Kis an object for control, a necessary device contra!
`may not be axeculed.
`
`Ogos}
`
`However, depending on the control content of the device on the moving object, even the same object may or may not be the
`
`
`contral object depending on the specific behavior of ihe oblect to the moving obleci. Far exampie, when the control content is an
`
`aulomatic braking function er an alarm function, when the preceding vehicie shows a oehavior traveling at a speed equal to or
`higher than that of the own vehicle, the preceding vehicle obstructs the traveling of the own vehicle Or since if
`is not an
`avoidance thing fo aveid, ii does noel become a cantrol object. On the other hand, when the oreceading vehicle traveling in frant of
`ihe awn vehicle shows a behavior of rapid deceleration ar shows a behavior of rapidly intruding into the front af the own vehicle,
`the preceding vehicis is an obstacie or obstacie that prevenis the traveling ofthe own vehicle it becomes an avoidance obiect to
`be avalned, and becornes an object fo be cantrolled such as the automatic brake funcion and the alarm function. Therefore, in
`order fo perform various contrals, iis desirable io be able to comprehend the specific behavioral content of the mabile abject.
`Although the three-dimensional chject detection devine disciosed in Patent Document 7 has described the detection of a moving
`ebiect shawn ina captured image, if does not touch on grasping the seecific Gehavier content of the moving oblect wHh respect
`to the moving object.
`
`roooe)
`
`Thea greseqdi invention has been made in view of the above background, and an Gbiect of the orasent invention is io oroperly
`identify whether or not an object existing around a mubile object is a control target. and fo mount the mobile object on the mobile
`object. His an chject of ihe oresent invention to provide a contra: object recognition device, a mobile device control system and
`8 codiral abject recognition program thal enable appropriate control af devices,
`
`roa?)
`
`in order to achieve the above obiect, the present invention comprises an object detection means for detecting a detection object
`present around ihe moving obiec! based an a captured image oGbainad Oy imaging the surroundings of the moving object, and
`the object A contro! target identification devices that identifies whether a detection target detected by a detection uniiis a contra!
`target that controls a predetermined device mounted on the mobile object, and iniages ihe periphery of the mobile object. The
`rigive oosition spectying means for specifying the relative pasion of the detection ablact detected by the object detection
`means with respect te the movable body based on the paralax information cbtained from the pair of captured images imaged by
`
`
`the giuralty of imaging means, and the imaging me Relative velocity specifying means for specifying the relative velocity of the
`
`detection ablect ia the moving object from ihe difference in the image position of ihe delection otfect between two or mare
`different captured images, and the relative postion identification means ft characterized by having a determining means which
`réiative speed is location and the relative velocity specification unit identified to determine whether or not the condition of the
`rigiive pasitian and the relive scead to be the coniral subject.
`
`(00s)
`
`if is possible to soppropriately identty whether ar nat an abject present around the moving
`According to the present invention,
`
`oblect is a conirel target, anc te achieve an excelled: effect of enabling appropriate control of a predetermined device mounted
`on the moving object. Fu.
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 3 of 13
`
`ooog]
`
`His a schematic diagram which shows schematic siructure ofa vahicle-mounted apparatus cantral system.
`
`itis a schematic diagram which shows schematic structure of the Imaging unit which comerises the vehicie-mounted apparatus
`control system, and an imaqge-analysis unit,
`
`itis a mode! eniarged view when the image sensor in the imaging part of the imaging unit is seen fram the direction orthogonal!
`to the Hight transmission direction.
`
`
`His a block dlagrarn showing compasiicn of a contral subject identification device in an embadiment. His a flow chan which
`
`shows 2 flow of identification processing of a control subject. (A) And (b) is a figure which shows the lefi-right image of a sterea
`camera, respectively. i is a figure exgiaining the principle of ranging calculation. tis an explanatory view showing an otect
`candidate Held set up in an image pick-up picture duminance picture). ft
`is explanatory drawing which shows the identificatian
`method of the oblect candidate area | region. (A) is the figure which showed typically the parallax image corresponding to the
`captured image shown in FG. (B) is a figure which showed typically the horzentai line parallax histogram Gf mam) obtained
`frora the parallax image of ine figure fa). (2) is explanatory drawing which Hlustrated the read surface area | region in ine
`parallax image of the figure fa).
`Ht
`is explanatory drawing which shows four types of feature pattern examples used for
`identification of a detection target object. H is explanatory drawing which shows the structure of the identifier used by the
`recognition process of a detection target object. (A) is explanatory drawing which shaws image area A, ©, E of each detection
`target chiect recognized by this object recognition process. (B} is explanatory drawing which shows tracking resull area | region
`A’, Gand § lof each detection target object in the following flame | frame. (A) is a figure which shows iygically the parallax
`image corresponding te the captured image duminance image} shown fo Fig.13 (4). (8) is a figure which shows typically the
`paraliax image corresponding to the captured image duminance image) shown in FRS.13 (bp). FIG. 6 is an explanotory diagram
`for descdbing siate feature quaniiies (XA, D2 A, VxA, Vy A, ¥ 2 A) ofa detection target Goreceading vehicie} displayed in an
`image area A.
`
`foo te}
`
`Hereinafter, an embodiment in which a cantrol object klentificaiion device according fo ihe present invention is used in an on-
`vehicie device cantrol system as a mobile device control sysiem will be described. FRG.
`1
`is a schematic view showing 2
`schematic configuration of an in-vehicie device control system. The in-vehicle device control system shown in FIGS. 4
`is provided
`wih an imaging uni 1OQ4 as an imaging unit thal images a region in fant of the traveling vehicle 190 In ihe traveling direction as
`
`an imaging region. The imaging unit 704 is installed, for example,
`in the vicinity of a rearview mirror (not shown) of the
`windshield 103 of the vehicle 146. Various dota such as captured image date obtained by imaging of the imaging unit 101 is
`
`
`input ig an image analysis un —_ 102 as an image processing uqdk. The inage analysis uqi 02 analyzes the data transmitted
`fram the iraaging un fi 101 to calculate the position, direction, and distance of the other vehicie present ahead of the host vehicie
`100, of on ihe road surface existing in the imaging region. Detect lane boundaries such as white lines.
`In the detection of
`another vehicle, an object on a road surface is deiected as 2 veticle based on a parallax image.
`
`ort
`
`The calculation result of the image analysis uni 72 is also sent io the vehicle travel control unt 1o4. The vehicie travel contral
`unt 104 drives ihe subject vehicle 700, far example, when ihe subject vehicle 700 is likely fo collide wih an abstacie, based on
`ihe detection results af the defection target such as a pedestrian or a traveling vehicle detected by the image analysis unit 102.
`informing @ person of a warning or performing driving Support control such as controtting the steering wheel and the brake of the
`wn verichs.
`
`{0042}
`
`FES. 2 is a schematic view showing a schematic configuration of the imaging unit 104 and the image analysis uni 702. As
`shown iq PYG. The imaging unl (O17 is @ stereo camera including hwo imaging unis 1104 and 1705, and the canfigurations of
`
`the two imaging unfs 1704 and 4108 are the same. The imaging unis 71OA and 1168 respectively include imaging lenses
`TTA and 1478, image sensors 1724 and 1128 In which imaging elements are hwo-dimensionally arranged, sensor substrates
`
`TIiSA and 7138 provided with the image sensors 1724 and T12B, and a sensor substrate Signal processing units 1744 and
`1145 that generate and outoul captured image data obtained by converting analog electrical signals (light reception amounts
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 4 of 13
`
`received by light receiving elements on the image sensors 1424 and 1128) output from 1134 and 1138 into dighal electrica!
`signals it consists of Luminance Image data is output fromthe imaging unit 194.
`
`10043]
`
`The imaging unii 107 aise includes = processing hardware unit 120 configured of an FPGA (Field-Programmabie Gate Array} or
`
`the like.
`in order to obtain parallax images from the iurninance image data output from the imaging unts TIGA and 1108, the
`processing hardware unit 120 calculates the caratlax values of corresponding image portions between the imaged images
`captured by the imaging units 170A and 110B. A disparity calculation unii t21 that calculates is orovided. The parallax value
`referred to here is a comparison iniage with respect to an image portion on the reference image corresponding to the same
`point in the imaging region, with one af the iriages taken by each of the imaging units 110A and 1108 as the reference image
`and the other as the comparison image. The positional displacement amount of the upper image portion is calculated as the
`
`
`parallax value of ihe image portion. By using the principle of tdangquiatien,
`if
`is passigie to calculate the distance from the
`parallax value ta the sare point in the imaging regien correspanding to the image portion.
`
`10044]
`
`Onihe other hand, the image anahysis unt 702 includes a memory 139 for sloring furinance image data and paralkix image
`
`data cuiput fram the ‘tnaaing uni ¢O7, and an MPU (Microcomputer that incorserates sofiware for perfarming identificatian
`
`nrocessing of control target oblects and parallax calculation control, Processing Uni 140. The MPU 140 executes identification
`
`
`sessing ofacontrol targ bject nding to the present embodiment g ihe luminance ima Jala and the paralla
`
`
`
`
`
`
`3ocessing
`of a cal
`arget o
`according ta the
`present embodiment usin
`ening
`wge
`data
`and
`the parallax
`image data stored in the memory £20.
`
`foots}
`
`FIG, 3 is a schernatic aniarged view of the image sensors 772A and 14123 when viewed from the direction arihagonal fo the light
`transmission direction. Fhe image sensors 112A and 112B are image sensors using a charge coupled device (OCD), a
`
`complementary metal oxide semiconductor (OMOS}, or the ke, and a photodiode tiga is used as an imaging element dight
`receiving element. The pholodiodes 112 4 are iwo-dimensionally anayed for each imaging pixel, and in order to increase the
`light collectian efficiency ofthe photodiodes 412 a, a micro fens 172 0 is provided on the incident side ofeach ghutodiode 112 a
`The image sensors 1124 and 112B are bonded io 4 printed wiring Goard (PYVB} by a method such as wire bonding fo form
`sensor subsirates 7234 and $132.
`
`Ogts}
`
`
`Next, cantrol object identification processing, which is a feature of the present invention, wil be deseribed. FRG. 4 is a block
`diagram showing ihe configuration of the contral dargel xieqtiicaiion apparaius of the present ernbadiment realized by t
`processing hardware unit 120 and the image analysis unl 102 in FIG. In the figure, the control object identification apparatus
`290 of the present embodiment includes a stereo image input unit 204, a paraliax Image calculation unit 202, a lurunance image
`igpul unt 203, an oblect candidate area recognition uri 204, an obiect racognitien unit 205, and an object iracking process. His
`configured to include a unit 208, an object state determination unit 207, a contro! target recognition unt 208, a recognition
`dichonary storage unt 209, and a recognition result output unit 270.
`
`10087}
`
`A stereo image is input fo the stereo image input unit 204 from a stereo camera (not shown) provided with an imaging unit
`having an imaging lens and an image sensor on the left and right. A luminance image which Is a left image ora right image of a
`stereo image is input te the luminance image inoul unlf 203 from ihe sharac image Inout unit 204, The input sterec image and
`luminance image are stered in the memory 130 of the sterea camera. The parallax image calculation unit 202 calculates a
`parallax value, which is 3 difference between the imaging positions af the left and right Images, using the stereo image input to
`
`ihe sterea image Input uni 204. and sets the parallax value as a pixel value. Caiculaie ihe made.
`
`Ootg}
`
`The abject candidate area recognition unit 204 detects a road surface area from the captured image using the parallax Image
`calculated by the parallax image caloulatien unt 202, and the defection reasudi of the road surface area and the luminance image
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 5 of 13
`
`input from the luminance image input unit 203 To set an ohiect candidate area incliding the detection target. The object
`recognition unit 205 perferms recognition processing of the detection target on the obiect candidate area set by the obect
`candidate area recognition unt 204. The object tracking processing unt 208 performs tracking processing on tne detection
`
`target abject
`
`rec:vegized by the obiect
`
`determination unif 2O7 uses the resull of tracking by the ablect tracking processing unit 206 te obtain a motion feature amount
`such as the relative cosition or relative velocity of the detection target.
`
`recognition unit 208 using the next
`
`luminance image frame. The object state
`
`ots}
`
`The recognition dictionary storage unit 200 is a mechine leaming methad such as SVM (Suppor Vector Machine}, and is
`dictionary data for recogniticn of a control target created using image sample learning data of the control target in advance
`
`{recognition dictionary Remember). The recagnitisn dictlanary storage unf 209 stores recagnition dictionaries ee
`
`created for each control ablect. The control target recognition unit 208 pedarrns cantrol target resagnition processing wit
`reference fo the recognition dictionary stored in the recognitien dictlonary storage unl 209 using the behavior feature amount
`obtained by the cbiect state determination unl 207... The recognition result output unit 270 cutouts the recognition result of the
`cantrol Gbject racognitian unt 208.
`
`og2a}
`
`Next, the operation of the object recognition device according io the present embadiment will bs described according to FIG. 5
`showing the operation flaw. A sterea image is Inout ia the stereo image Inout unk 201 of FRG. 4 (3404). Specifically, a sterea
`image is Inout fram a stereo camera. FIG. 6 shows an example of a stereo image. In stereo images, the same subject is imaged
`ai different imaging positions by the left and right image sensors. Then, the luminance image Input unt 209 of Fic. 4 inputs a
`luminance image (refarance image) of eliher the lef image or ihe right image of ine stereo image (S192). The input stereo
`image and luminance image are stored in the memory 130 of the stereo camera.
`
`os]
`
`The oblect candulate area recognition unt 204 seis an object candidate area ina place where an cblect exisis based on the
`luminance image (2103). Specifically, first,
`in order to set a candidate region of a detection target, as shown in FIG. & a
`reclangular Glock is set according io the image of the detection iarget in the captured image. The sosition of the rectangular
`
`blockin the captured image and ihe she of ihe rectangular block are specified by the upperleft coordinates Ofs, Ys} and lower
`
`fight coardinates (Xe, Ye) of the rectangle as shown in Fic. When seiting rectangular blocks, the rectangular blacks are set in
`
`
`order fromthe largest rectangular block. This rectangular block is an Gbiect candidate area.
`
`10022]
`
`The parallax image calculation unit 202 calculates the carallax value which is the difference between the imaging positions of
`the left and right images in ihe target frame using the stereo image input te the stereo image input unit 207 (6104), Specifically,
`for ihe same cortion af the sterac image formed oy the lef and right imading lanses, the parallax value is calculated by ihe block
`matching method, and the parallax image with the calculated parallax value as the olxel value is calculated. The block matching
`method is a method of dividing left and night images into blocks, and when the bisok similarity in the left and right images is the
`
`largest, oolaining a paralax value from a portien where the blocks are muiched. For exarnpis, a 1280 « 960 pixel image is
`divided info 5 x & sized blocks. The optimal value of block size is adjusted and set by experiment. in the examole shown in FRG.
`
`7, the imaging positions tn the lef and night images with respect fo the paint © on the subject are the distances Ad and Az from
`
`the imaging center, Therefore, the parallax value A is calculated as A = Ad + Ae. Therefore, the parallax image is an image
`having parallax values at pixel positions.
`
`(0023)
`
`
`The ogiect candidate area racognitien unit 204 creates a horizontal line parallax histograrn (¥ mag) showniq FIG. 105 using the
`parallax image obtained by the parallax image calculation unit 202 (S405). The horzontal line parallax histogram js a histugram
`
`obtained by integrating the parallax values included in ihe parallax Image in one horizontal line of the parailiax image in FRG.
`
`That is, FG. 10 (6) shows the frequency value distributiog H CA, y) al the paraiiax value A. The horizontal axis of FG. TOG fs the
`parallax value A, and the vertical axis is the height y of the parallax image. The value of each pixel of the horizontal line parallax
`
`histogram is the frequency of the parallax value of oneling of the height y af the parallax image.
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 6 of 13
`
`foo24)
`
`Tha road surface area projeciad oy ihe cagturad image tends to decrease in width at a suostantially constant rate from the lower
`par to the upper cart of the image. Then, the road surface partion shawn on the same horizontal line on the captured image has
`substantially the same parallax value A on the parallax image because the distance fram the host vehicle 190 is substantially
`
`ihe same, Therefore, the gixels corresponding to the road surface area have a Hnear distrigution having an inclinaiion as shown
`in FIG, 198 on the horizontal line garallax histogram.
`
`{0025}
`
`line parallax
`Therefore, ihe otfect candidaie area recognition unit 204 firsi detects pixels linearly arranged on the horkmonial
`histogram, and mags the detected pixels on the parallax image. Thal is, the pixels Hnearly arranged in the horizontal
`ti
`
`paraiiax histogram are associated as pixels of the road surface area on the parallax image. Thus,
`if the pixel which projects a
`mad sudiace is known, a road sundiace area 302 as shown in FRG.19 ©) can be detested by perfarming an interoolation process
`between the pixels (8106).
`
`26}
`
`Specifically, the ablect candidate araa recogrilian unit 204 accuiras: paraiiax distributien infarmation in aach harizontal directian
`fram ihe parallax image calculation unit 202, and the pixel distribution on the horizontal line parallax histegram specified fram
`the information is obtained by the least squares method of the existing technology or Linear approximation is performed by the
`Haugh transform method. An approximate siraight line as shown in FIG. 108 ablained by this becomes a straight line having
`such an inclination that the parallax value becomes smatiier toward the upper side of the image in the horizontal line parallax
`histogram corresponding to the lower part ofthe parallax image. The oixels distributed on the approximate straight line or in the
`vicinHy thereof are present al substantially the same distance in each hodzontal line on ihe parallax image and have a high
`occupancy ratic, and a target of which the distance is continuausly farther away foward the unper side of the image Ht is a pixel.
`That is, ihe pixeis distibuted on the approximate straight line in the horizontal line parallax histogram or in the vicinity thereof
`incicate thai the roaci surface area is projected, As a result, ihe mac suriace area S02 can be detected by specifying the pixels
`
`distributed on the approximate straight line in the horizontal Hne parallax histogram or in the vicinity thereof, Although the road
`surface aréa is detected by the above method, the road shoulder and ihe white line are recognized framthe luminance image,
`and the road surface ares is detected by specifying the continuous area of the road shoulder and the area where the white Hne
`exists. You may
`
`{0627}
`
`Here, inthe present embodiment, an opstacie that hinders the traveling of the hast vehicie 100, an object fo be avaided, and the
`like are detected, and control is performed ic notify the driver of the host vehicle 100 of a warning. And driving support contra!
`
`
`such as controling ihe brakes. An oblect (contral oblect) to be detected fo realize these controlsis an object present on ihe road
`surface on which the vehicle 190 is traveling.
`
`{0028}
`
`Therefore, in the present embodiment, when ihe road surface area 302 is detected, the obiect candidate area recognition unit
`
`204 sets the recognition weight of ine road area ta 1 and the recognition weight of the other image areas fo 0 (S707). Then, an
`oblect candidate area overlapping the area whose recognition weight is 1
`is an oblect candidate area which is described later on
`the road surface as an object candklate area existing on the road surface, while an ohiect not ovedapping the area whose
`recognition weight is ¢ The oblect recognifion process described later is nat pedurmed on the candidate area as the candidate
`area of the object not existing on the road surface (S108). Therefore,
`in the example shawn in FES. 8c, object recognition
`precessing is gerformed on object candidaie regians indicaled by reference numerais A, C, and ©, and abject recagnition
`processing is performed on object candidate regians indicated by reference numerais B,D, and F.
`iH dues not happen. As a
`result, in addition to the possibility of reducing false recogniticn In which an obiect that is not the contra! farget is errongously
`recognized as the cantrol hargei, the qurnberof object candidate areas for which the oblect recognition processing is pedormed
`is reduced. .
`
`ooegi
`
`http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval& COUNTRY... 6/9/2019
`
`

`

`2015133078-Desc-en
`
`Page 7 of 13
`
`The oblect recognition uni 205 perferms an ohiect recognition process of the detection target on an obiect candidate region
`overlapping ihe region having a recognition weight of ¢ among the object candidate regions set by the cbiect candidate region
`recognition unit 204 ($109).
`in this oblect recognition process, for example, a recegniiion dictlonary is created using image
`sampie learning data on which an object is recognized in advance by a machine learning method. Af this time, it is necessary to
`create recagnition dictionaries separately depending on the detection langet.
`
`(0030)
`
`First, the obiect recognition unit 205 sets the white region 404 and the black region 402 In the four types of black and white
`
`blocks Ato D as shown in FIG. Tf for the object candidate region set by the object candidate region recognition unit 204 (see
`FES. Calculate the feature quaniities of Undicated by middle hatching). Specifically, the surn of ihe pixe! values for the oorion in
`the obiect candidate area corresponding to the white area 401 of cach black and white block 400 of Ato 5 as shown in FRG. 11
`and the abject candioaie area carresconding to the black area 402 Calculate the sum of pixe! values for the lacation of. Then, a
`value obtained by subtracting the oixel value sumin the black area 402 from the gixe! value sum in the white area 407 is set as
`
`the feature amount h Og regarding ihe type of black and white block for the object candidate area.
`
`10034]
`
`One evaluation value f Gd is calculated from the feature amount h Og calculated in this manner, using the evaluation functian
`shown in the following equation (1). The evaluation value f Od is a feature amount ht Gg} for cach black and white block @ = 4 to
`¥ (Poo 4 in fhe case of using four black and while blocks Hhuistraied in FIG. f¢}}. To calculate by surmrning the weighted feature
`quantities a t= ht Og of the respective black and white blacks obtained by mullipiying the weight calculation results by the
`weighting factor at given for each black and white block. te can. Here, the feature amount ht OO and the weighting factor o t can
`be obtained by collecting learning data for the luminance image of the detection target and leaming fram the learning dala.
`
`£00223
`
`
`As shown in FiG. 12, the classiier used in the recognition process of such a detection target is configured to include a plura
`of levers 500-4 ion 1 is 4 positive inleged. Each of the layers 500-7 io 500-n has individual evaluation functions using differant
`
`feature sets features h t &) and weighting factors at) that are characterized as being specific detection abiects. ing. Then, for
`the object candidate area, first, an evaluation va