ii)»; «200% E 13393
`
`PATENT ABSTRACTS OF JAPAN
`
`{i Upubiication number :
`
`2004‘“? 73973
`
`{43:3Bate- 04F pubiication 01° appiicatiofi : 24.06.2034
`
`{5 i )int‘Ci.
`
`ABIB UM
`
`£3028 23f24i
`
`H04?! 1” 3
`
`(2i)Appiication number : 2002~344395
`{22)Date of fiiing :
`233.131.2302
`
`{T iiAppiigant : AZEMOTO SHIYOUGG
`(?2}inventor : YANAGUCHI TETSUZD
`NAKAMOTO MASAHIKQ
`NAKAJIMA YOSHIKAZU
`SATQ YOSHNOBU
`HASHIZUME MAKGTO
`TAML‘RA SHiNICHI
`SUGANO NOBUHIKO
`
`{54} METHOD FOR ESTIMATING PARAMETER 0F PRDJECTION MODEL FOR STRABISMUS
`ENDOSCDPE
`
`{§?}Abstractt
`PROSLEM TC} 8E SDLVED: To provide an extended mangew
`ofireaiity operation support system corresponding to a
`strabismus Endoscopic».
`SOLUTTON: A pasition marker is formed in an insertion part
`12 of the atrabismus endoscase 10. The position of the,
`pasition marker in a warid coordinating systam is measured
`whenever the insertion part i2 is mutated, and the parameters
`rm: and cw {ii a projection mudei are estimatad based on the
`measured vaiues of the pasitian. Next, the parameters me and
`at: of this: prejectian: modai are estimated based on that
`coordinate value in the image coardinate system of a
`projectian paint from time worid canrdinate system to the
`image coordinate system Dbtained by the projactien modei
`according to the estimated parameters rm and cw, and the
`the coordinate vaiue actuaiiy measured in the image
`
`canrdinate system (if the: projectimn point.
`
`* NOTICES =i<
`
`JPQ and INF}? am not respansihie for any
`damages caused by the use of this transiatian.
`
`‘¥.This document has been transiated by computan So the transiation may not refifict the originai
`preciseiy.
`2.**** shews the word which can not be transiateci.
`
`31m the drawings, any wards are: not transiatedi
`
`

`

`3?»; «200% E 13393
`
`{"3
`
`{(3iain1isii
`{Cieirn 1]
`An insert portion extended to iinesr shops inserted into the sodominei cavity, and an observation port
`provided by tip surface of this insert portion, Have an imaging surface where an image of an observing
`object is imaged and an optic axis of the aforementioned observation port inciinos to an axis iine of
`the aforementionoci insert portion, and the aforementioned insert portion receives the aforementioned
`imaging surface, image processing system ”RH for stre‘oism endoscopes which processes on image
`acquired by a strabism endoscope reiativeiy constituted pivotebie considering the aforementioned axis
`iine as the axis of rotation“ it is how to presume o parameter of a projection modoi specified by a
`iower type {i} which determines conversion in an im‘ageccoordinstes system sci; as the
`aforementioned imaging surface from a worio cooroinsto system set as exterior space of the
`aforementioned strahism endoscope
`3i02MT8TRotG{“th5-3tav no, cc) Tnmnitheta, no: Cw) on c“ Ci}
`(Here, a vector and is showing a position of a point on the aforementioned worio coordinate system
`ow} A vector anti nw showing a position of a point on the aforementioned imagewccordinates system, A
`vector and Cw which specify the direction of the aforementioned axis iine in the aforementioned worid
`coordinate system, A vector and no which specify a passing position of the aforementioned axis line in
`the aforementioned worid coordinate system, A vector and cg which specify the direction of the
`aforementioned optic axis in the aforementioned camera coordinate system, A vector and theta which
`specify a passing position of the aforementioned optic axis in the aforementioned corners cooroinste
`system, An angie of rotation when the aforementioned insert portion sets the axis of rotation as the
`aforementioned axis fine and it rotates reiativeiy to the aforementioned imaging surface, and To,
`Conversion and Tamra showing correspondence to the aforementioned camera coordinate system
`when the aforementioned insert portion sets the axis of rotation as the aforementioned axis iine and
`it is not rotating reiativeiy to the aforementioned imaging surface from the aforementioned worid
`coordinate system. Conversion and TRotC showing this rotation when the aforementioned insert
`portion sets the axis of rotation as the aforementioned axis iino and it rotates reistivsiy to the
`aforementioned imaging surface in the aforementioned worici coordinate system, Gonversion and M
`showing Spin compensation within the aforementioned camera coordinate system show an internei
`parameter pecuiiar to the aforementioned strahism endoscope, and 8 shows a predetermined scaiar,
`respocfivowo
`A piece marker which moves with rotation of a circumference of the aforementioned axis iine of the
`aforementioned insert portion in a centering on this axis iine circular orbit top is provided Trio! which
`rotates the aforementioned insert portion reiativeiy to the afbrementioned imaging surface by setting
`the axis of rotation as the aforementioned axis iine A moitipiectimes deco, Each point estimate of
`vector “w of an upper tyoe {’1}, vector cw, and the angie of rotation theta is csicuioted by measuring a
`position in the aforementioned worid coordinate system of the aforementioned piece marker for this
`every triai, and specifying the aforementioneo circuiar orbit based on each measured voice of this
`position:
`Next, a cooroinst‘e value in this image—coordinates system of a projected point From the
`aforementioned worid coordinate system to the aforementioned imagemcooroinetcs system produced
`by substituting each point estimate of the sforemen‘tionedi vector ow. vector cw, and the angie of
`rotation theta for an uooer type {i}, A parameter estimation method of a projection modei for
`strabisrn endoscopes characterized by csiculating each point estimate of vector ns of an upper type
`{13:3, and vector or; based on a coordinate veins surveyed in this im‘ageccooroinatos system of this
`projected point.
`
`{Transiation done}
`
`* NOTICES *
`
`INF}? are not responsioie for any .
`JPG anti
`damages caused by the use of this transietion.
`
`iffhis document has been transieted by computer. So the transiation may not reflect the originai
`precisciy.‘
`finance: snows the word which can not be transiated.
`
`Sin the drawings, any words are not transieteo.
`
`

`

`ii)»; «200% l 13397:?3
`
`Lu
`
`DETAILED DESCRIPTION
`
`{Detailed Description of the invention}
`{3301]
`{Field of the invention}
`The present invention relates to the parameter estimation method of the projection model for
`strabism endoscopes in the image processing system for strsbism endoscopes which processes the
`image acquired by the strebism endoscope.
`{0002]
`{Description of the Prior Art}
`About the endosoooic operation excellent in low invsslng nature, the extended senserol’wreellty
`operation support system of former versatility is developed and disclosed {refer to the following
`nonpetent literature no).
`inventors of the present application also develop the extended senserof—reelity leperosoopio surgery
`supporting: system using :3 threemdimensionei oltrasonogrsohic image, and are disclosing this (the
`following see nonpstent literature l3},
`{0003]
`Although it is a system for en aoceptlngrreslity endoscope {endoscope with whieh the optic axis of an
`observation port ioohs at the axis line oi an insert portion, and the transverse piano which is
`corresponding), these operation support systems“ in an endoscopic operation, in order to secure as
`wide View, a strsbism endoscope {endoscope with whioh the optic axis of an observation port inclines
`E0 the] axis line of an insert portion and which looks at an oblique direction) is useol in many oases.
`
`{3804
`
`in the strehiem endoscope, an optical device, such as using combining a widersngle lens or swing
`prism, is performed? so that a wide View can be secured, in order to realize an extended sense~otr
`reality operation support system corresponding to the strehism mirror in which suoh an optical device
`was mess, The projection model which determines the conversion in en imageroooroinates system
`(coordinate system set to imaging surfaces. sooh as CCD with which the strslaism endoscope was
`equipped, etc) from a world coordinate system {coordinate system used as the standard set as
`threefldimensionsl space), The model of the external parameter which determines the conversion to
`the camera coordinate system which is needed in the projection mooei from a world coordinate
`system, and an internal parameter peculiar to a strstiisrn endoscope and the technique of presuming
`them are needed, it is possibie to apply the motlei for acceptingwreelity enooscoces about an internal
`parameter. and it is possible to use the technique {for example, the following nonpatent literature 11,
`32 references) of the conventional prooosai being made also about the presumption. However. about
`the external parameter, the model which can respond to e strshism endoscope until now was not
`proposed; and, for this reason, that estimation method was not proposed. either. By such is situation,
`
`Elie pliojeotion model corresponding to a strabism endoscope was not known conventionally,
`
`3005
`About presumption of the distortion parameter of a widensngie lens, many highly precise techniques
`are already proposed {refer to the following nonpetent literature 8mm}. Also in research of e strehism
`endoscope, the technique of synthesizing and forming the View of each movement position into a false
`wide angle on the conditions of fixing motion velocity of e strahism endoscope is proposed {the
`following see nonp‘etent literature 7}. However, it cannot apply to the extended sensenofwreeiity
`operation support system corresponding to e strehlsm endoscope,
`looms]
`{Nonpetent literature l]
`D. Devi D‘Gobbi, F’Siomlta, KJMSurry. and TMPeters, and "Automatic fusionmof freehand endoscopic
`brain images. to threemdimensionsl surfaceszoreeting stereoscopic psnorsmes.",lEEE
`TrensMedJmangol. 2i {pp23‘302002
`{900'}?
`{Nonpatent literature 2]
`.J. Heiferty, CéZhsngi GiMoLennani and WHiggins, and "Videoendoscopic distortion oorreotioneno its
`application, to virtual guidance oiendoscopy",iEEE Trans.Medlmesflol{ZGiopifififirfil6.2002.
`loose]
`{Nonpatent literature 3]
`Si), Beck et el, and "A system. to Support leperosoooio Surgeryby augmented reality
`visualization.“ Springer—VeringOGl Proceedings,MlCCAl,pp.89'l “638
`{0009]
`{Nonpatent literature 4]
`Rs Kheclem et el, and ”Endoscope, calibration anosocuracy testingl‘or SDEZD image
`rglstration."Springer~4‘iferleg,2i301.Prooeeizlingei MlCCAippii 361'“? 365
`
`

`

`ii)»; «200$ E 13393
`
`n n
`
`{Ofl’lflj
`{Nor‘xpatent literature 5]
`RJ. Edwards, APKlng. QRMaurer, D.A.de Dunne. DJHawkers. {31.13. Hill} R.P.Gastion, MRFenlon.
`A.J.usczyzek. A.J.Strong, CL. Chandler. and Mdfiieeson, "Design one Evaluation of a system for
`microseeperassisted guided interventions WAGE"
`{Gilli l]
`{Neopatent literature 6]
`JD. Stefsnsic. AJHerline, “(Shana wcflnamen, J.M.Fitzoatrini<, 8M. Dawent. and R.L.Qalioway, and
`"Registration. of. ohysioei. eoa’oe toleparoeconic. image space for use in minimally invasive hepatic
`surgery”.lEEE Trans.Med.im:ag.,voi.20,pp. Nil 2% OE3£GUQ
`{GO‘lEJ
`{Nonpatent literature 7]
`H. Formation { of the endoscope View by Yameuchi et at. and
`wide angle. JLJSCAS. voi.2, no.2. and po.fi£~§8*2000.
`isms]
`{Neopatent literature 8]
`H. Heneishi, YYegihasni, and and. Y. Miyake"A Newmetnod. for distortion correction of electronic
`endoscopic images".iEEE Trams. Med.Imag..voi.i4.No.3,pp.548-555,i9‘95.
`{0014]
`{Neopatent literature 9]
`K. Viiayen et at and? ”A. new aooroach for nonlinear. distortionoorreotion in endosoooic images based
`on least squares estimation,"iEEETraneMsciimag.Natl8,pp.345~354.l995.
`{0015]
`{Nonpatent literature i0]
`WE. Smith et al, "Correction of distortionln endoscopic images. "lEEE TransMedlmag, Vol.11,
`pp} i7w’l22~l 9‘92.
`{0018]
`{Nonpatent literature i l]
`Wang and WHTsai and "Camera. calibration by vanishing lines for 3%) computer
`vision",IEEETrensPAMLx-volj3,rwo.4.pp.370~376, 189i.
`{Qflifl
`{Monomers}: literature 'i2]
`RY. Teal and ”A versatile. camera calibration technique, for nignrasourecy 3%). machine vision
`
`endosoooe to memorize” 3 ”of false
`
`{metrollogy‘ using ofrathrsneif tv oemsreeand lsnsee".iEEE J.Robot.Automet..voi.3.op.323~344,l98?.
`ems
`{Nonpatent literature l3]
`M. Nakamoto et al. and "35. filtrasound System Using. a Magnetomflptlc Hybrid. Tracker for
`Augmented. Reaiity Visusiiaation in LaparosconicLiver Surgery".in MECCA? 2002,LNCS 2489,pp.l48*
`iSfiSeptember 2002.
`isms]
`{Probiem to be soiveo by the invention}
`As mentioned above, the extended sense~ofr~reality operation support system corresponding to a
`strabisrn endoscooe was not realized until now. Rowever, especialiy‘, in intraneritoneal. joint shaping,
`and an otoiaryngoiogy operation, since a strabism endoscope was used in many cases. to reailze the
`extended sensewof‘reaiity operation support system corresponding to a strahism endoscope was
`desired strongiy in recent years.
`Then, in order that an inventor of the present. application might realize the extended sensswofwreeiity
`operation support system corresponding to a strahism endoscope, it developed the image omoessing
`system for strabism endoscopes provided with the projection model corresponding to a strabism
`
`sisncioejcope. and disoioeed this in the Patent Office on an application concerned and the some day.
`
`0820
`
`An object of the present invention in order to realize the extended senseeofereality operation suoport
`system corresponding to a strabism endoscope is to proviso the parameter estimation method of the
`
`{inrojecjtion model for ** in strabiem in this image processing system for strabiern endoscopes.
`002i
`{Means for solving probiem}
`In order to soive an aforementioned problem the parameter estimation method of the projection model
`for etrebism endosoopee of the present invention, The insert portion extended to the linear shape
`inserted into the abdominal cavity, and the observation port provided by the tin surface of this insert
`portion. Have an imaging surface where the image of an observing objent is imaged. and the optic: axis
`of the aforementioned observation port inoiines to the axis line of the aforementioned insert portion,
`and the aforementioned insert portion receives the aforementioned? imaging surface. image processing
`system **** for strabiem endoscopes which processes the image acquired by the strabiem
`
`

`

`ii’w «200-414 :393‘3
`
`U1
`
`endoscope roiativciy constituted pivotabie ooncidaring the aforementioned axis iinc as the axis of
`rotation It is how to presume the parameter of the projection modoi specified by the Eowor type (2}
`which determines conversion in the imagerooordinatcs system set as the aforementioned imaging
`surfaoo from the worid coordinate spat-om act as, tho oxtorior space of the aforementioned straoiam
`endoscopic,
`5503MTQTRgitC{”thEtE, no, co) Tnmvitneta, nw, ow} on W (2}
`(Hero, tho vector and is showing the position of the point on the aforementionao worid coordinate
`ayotom on) The: vector and Uw showing tho position of the point on the aforementioned camera
`coordinate system, Tho vector and? cW which specify the direction of the aforementioned axis iino in
`the aforementioned worid coordinate system, The vector and no which specify tho passing position of
`the aforementioned axis iinc in tho aforementioned world coordinate system, The vector and co which
`specify the dirootion of the aforementioned optic axis in the: aforementioned camera coordinate
`system, The vector and theta which specify the passing position of the aforementioned optic axis in
`the aforementioned camera coordinate system, An angina of rotation when the aforomontioned insert
`portion sets the axis of rotation as the aforementioned axis iine and it rotates rciativeiy to the
`aforementioned imaging suriaoe‘ and To The conversion and Tnoiw showing the correspondence to the
`aforementioned camera coordinate system when the aforementioned insert portion sets tho axis of
`rotation as the aforomantionoo axis lino and it is not rotating roiativaiy to the aforomontionod imaging
`surface from the aforementioned worid coordinate: system, The conversion and TRotG showing this
`rotation when the aforementioned insert portion sets the axis of rotation as the aforementioned axis
`iine and it rotates roiativoiy‘ to the aforementioned imaging surface in tho aforementioned worid
`coordinate system. The oonvarsion and M showing the spin compensation within the aforementioned
`oamcro coordinate system Show an internal parameter peculiar to the aforementioned strabism
`
`Eandosjcopo, and 5 shows the predetermined soaiar, rosoootivoiy.
`
`0022'
`
`Tho piaoe marker which moves with rotation of the circumference of the aforementioned axis fine of
`the aforementioned insert portion in a centering on this axis iino cirouiar orbit top is provided. The
`triai which rotates the aforementioned insert portion roiativaiy to the aforementioned imaging surface
`by setting the axis of rotation as the aforementioned axis fine A mui‘tipiortimos dead. Each point
`estimate of vector 31w of an upper typo {2}, vector CW, and the angic of rotation theta is caicuiatcd by
`moaouring the position in the. aforementioned worid coordinate system of the aforementioned piaca
`marker for this every triai. and specifying the aforementioned circuiar orbit based on each measured
`vaiue of this position,
`Next, the cooroinato vaiue in this image—'ooordinatos system of the projected point from the
`aforementioned wcrid cocrcisnato system to the aforementioned imagaucoordinotos system produced
`by substituting each point estimate of the aforementioned vector nw, vector cw, and tho angie of
`rotation theta for an upper type (2}, Basad on: the coordinate vaiue surveyed in this image:w
`coordinates system of this; projected? point, each point ostimato of vector nc oi an upper type {i} and
`vector on is oaiouiated.
`{3023]
`{Mode for carrying out the invention}
`Horoafier, it describes about the embodiment of the present invention using Br‘awings.
`{0024]
`< Straoism endoscope apparatus outfino)
`
`i
`“is a figure showing an oxampia of a ctrabism endosoopc apparatus tyoioaiiy Trio iiiustratco
`otrahisrn endoscope apparatus i is provided with the strabism endoscope (straoism type rigid
`
`fnooslcopo} it}, theimage processing system 20 and theimage dispioy device 33.
`
`$3025
`
`The strabicm endoscope i0 is provided with the foiiowing.
`The camera part H providco with imaging surface {not shown) constituted by image sensors, such as
`(305, eto‘
`The hard insert portion 12 extended in the axis Zine Lw direction from this camera part N,
`This insert portion 12 is a portion inserted into the obdominai cavity etch, and that tip surface iEa is
`formed so that it may cross diagonaiiy with the aforementioned axis {inc LW. The insert portion i2
`sets the axis of rotation: as the aforementioned axis iino Ln, it is constituted pivotabio to the
`aforementioned oamora part, and the ohsorvation port 13 is provided by the tip surface 326:. Optior
`axis LC of this observation port 13 is constituted so that it may inciine to the aforementioned axis fine
`Lo, and if the: aforementioned insert portion 12 is rotated by setting the axis of rotation as this axis
`iine Lw, it wiii rotate this opticraxis. Lo roiativoiy to the: aforementioned imaging surface. The
`aforementioned axis iino Lw is defined in worid coordinate system {coordinate system set as exterior
`space of strabism endoscope 10} SW: and the aforementioned ootiooaxis LC. is defined in camera
`Coordinate system {coordinate system set an observation port 13} Sow
`
`

`

`ii)»; «20044$933
`
`{3
`
`{0926]
`in the aforementioned strshism endoscope apparatus ii the reflected light from an observing object is
`incorporated in the insert portion i2 from the observation oort i3. and is imaged in the image sensor
`in the camera part ii. And in the image processing system 20‘ signsi processing of the picture signai
`acquired by the image sensor is carried out. the picture signei by which signs! processing was carried
`fut ftarthfif is converted to an image in the image oispiay device 30, and it is oisoiayed.
`(3927
`< The image processing system for strshism endoscopes}
`The aforementioned strabism endoscope apparatus 1
`is provided with the image pressesing system
`for strebisrn endoscopes This image processing system for straoism endoscopes processes the
`picture Signet acquired by the strsoism endoscope iii}, forms the image which can be used for
`extended sensecof‘rsaiity operation support, and is constituted by an arithmetic circuit. on ooeretion
`fggggim etc. which were orovided in the image dispiay device 30.
`And this image processing system for strsbism endoscopes is provided with the projection modoi
`which determines conversion in the image-“coordinates system set as the aforementioned imaging
`surface from the aforementioned worid coordinate system SW, This projection modoi is prescribed by
`the iower type {3} using interns? parameter M oeoniisr to the aforementioned strahism endoscope it},
`and externoi parameter T which determines the conversion to the aforementioned camera coordinate
`system So from the aforementioned world coordinate system SW.
`{0029]
`sicxMpr
`{0830]
`Here, the vector as which on expresses the position of the point on the aforementioned wcrid
`coordinate system, the vector as which in expresses the position of the point on the aforementioned
`
`(3}
`
`Earrings coordinate system and 5 show the predetermined soaiar, resosmosis»~
`
`003];
`
`{43
`
`This image processing system for strsbism endoscopes has specified the oforcmentioneci external
`parameter T by the lower type {4}.
`{0932]
`“in'i'g'i'ogig;(*theta= no. cc} Tomi: {theta nw, cw)
`{0933]
`The vector and cw which specify the direction of the aforementioned axis iine in the aforementioned
`worid coordinate system here “w, The vector and no which specify the passing position of the
`aforementioned axis fine in the aforementioned wot-id coordinate system. The vector and on which
`specify the direction of the atorementioned optic axis in the aforementionec} camera coordinate
`system The vector and theta which specify the passing oosition of the aforementioned optic axis in
`the aiorementioncn camera coordinate system, An engie of rotation when the aforementioned insert
`portion sets the axis of rotation as the aforementioned axis tins and it rotates reietivsiy to the
`aforementioned imaging surface, and To, The conversion and Tnmw showing the correspondence to the
`aforementioned camera coordinate system when the aforementioned insert portion sets the axis of
`rotation as the aforementioned axis line and it is not rotating roiotiveiy to the aforementioned imaging
`surface from the aforementioned worid coordinate system. in the aforementioned worio coordinate
`system, the aforementioned insert portion sets the axis of rotation as the aforementioned axis tine,
`and conversion showing this rotation when rotating reictiveiy to the aforementioned imaging surface,
`and Time show conversion showing the shin oomoensation within the aforementioned camera
`coordinate system, respectiveiy,
`{3034]
`<1” The projection modei of e strahism endoscope}
`
`
`Hereafter, it describesin detaii about the aforementioned projection model using ‘" "land Fig.3..f
`
`
`and Fig 3 are the figures showing the concept of a projection modei typicaiiyi:
`shows the
`conversion to the aforementioned camera coordinate system Sc from the aforementioned worid
`coordinate system SW, and Fig.3 shows the coin compensation in the aforementioned camera
`coordinate system Sc.
`{0035]
`The position vector of the point of the aforementioned? worid coordinate system 3w Pea" (no. yw. aw,
`i}. if rigid? body affine transformation {it is equivaient to the aforementioned externai parameter T}
`which makes Po the position vector of the point of the aforementioned camera coordinate system So,
`and is converted to camera coordinate system So from worio‘: coordinate system So is set to T, The
`conversion to Po from: PW is nonoted by a iower tyne {5}“
`{0036]
`oozTow
`
`{5}
`
`

`

`ii’w «200-434 :393‘3
`
`{003'}?
`
`{Exterrjiei parameter T in an ooper type {53 is denoted by a icwer type (6}.
`
`.0038
`{Mathematicai formuia 13
`
`,
`
`“R t
`
`Taxi,
`‘\ ii
`
`Y
`
`Jere-can:
`
`‘
`
`‘
`
`{63
`
`1
`
`{0039]
`Here. R expresses threesiine the orthogonai matrix of three rows which means rotation. i: expresses
`Ehe tiareerdimeneionoi vector meaning ooraiiei treneiation, and i expresses the identity matrix.
`{3MB
`On the other handi if the position vector of the twoudimensionai orojected point of the
`aforementioned PC is made into i523" (uofixox’zci vofiycfzc. i}. The correspondency From this projected
`point ip to image point is corresponding with this projected point ip on the aforementioned image“
`coordinates system is denoted by a lower type {7"} using the aforementioned internei parameter M,
`{cost-i]
`sioii‘viin
`{0042]
`Here. is shows the predetermined soaiar.
`{0343]
`
`(3’)
`
`Enternjsi parameter Ni in an upper type {7) is denoted by a flower tyoe {83‘
`
`{3044
`
`{Mathematicai Formuia 23
`
`«'F ix,”
`Rim {3
`F}
`
`{3
`f it?
`{3
`
`11,3
`it
`i.
`
`'
`
`- (83
`
`{0045]
`Here. F expresses a Focai distance. kc and? RV express the unit iength of 1.: axis and v axis, resoeotiveiyi
`and 3 (cc V0} expresses the coordinates of the image center.
`{0046]
`in the aforementioned strebism endoscope '20 the aforementioned external parameter T can be
`
`described as Foiiows First as shownin V“ 1333 in the aforementioned strsbism endoscope i0
`conversion to So this camera coordinate system 5. when the aforementioned camera coordinate
`system So is not rotating to the aforementioneo woriizi coordinate system 8w From this worid
`
`
`“ 33.3333 it writes See [the camera coordinate system by which
`
`this rotation is not accompanied ,3,
`{384?}
`Subsecuentiy as shownin {M11333 in worio coordinate system 5w. the case where oniy the engie
`theta rotates the aforementioned insert, portion 32 to the aforementioned imaging surface is assumed
`by setting the axis of rotation as the aforementioned axis iine Lw Conversion showing the rotation at
`this time is made into Thom; (theta, ow CW3 The aforementioned axis has Lwis prescribed by the two
`
`{new38km [the camera coordinate system whichgave conversion of the aforementioned To 3.
`
`0948
`
`Camera coordinate system so; after the aforementioned rotation is rotating the camera coordinate
`system before rotation reiativeiy to See) 0n the other hand, the position over the aforementioned
`worici coordinate system Sw of the imaging surface (and the aforementionec imagewcoordinates
`system3 of the aforementioned stiahism endoscope ”i0 is not changing For this reason the coin
`compensation in a camera coordinate system is needed in the present invention as shownhim‘
`i
`
`the axis oF rotation is set E the eForemenFioned camera coordinate syetem 3 as the aForementioneci
`
`

`

`ii)»; «200% E 13393
`
`g
`
`opticnoxis La for She. The aforementioned hand of out is an: angle to a counter direction. *' This spin
`compensation is peiiormed by giving conversion Tonto C“ theta. no. Go} whioi‘i carries out rotation
`WWW only of theta,
`isms]
`By apoiying the projection mono! containing external parameter T specified oy performing suoh a
`series of conversion {4} is, an upper typo specified by the upper type {3}, It becomes possible to
`determine conversion in the imagomooordinates system set as the aforementioned imaging surface
`from the aforomontioned worid coordinate system Sw.
`{@0501
`< Parameter ostimation method)
`
`Next, it descrioos about tho oaramster estimation method of the projection modoi for strabism
`endosoooos concerning one embodiment of the prosent invention. First, presumption of intemsi
`parameter M is performed with conversion To in case the shoveNmontioned rotation does not ioiiow. it:
`is possible to appiy the various techniques of the conventional prooosai boing made as those
`estimation methods. Also in this embodiment, orosumotion of the: aforementionod conversion To and
`the aforementioned internal parameter M is oerformoo using the oonventionai technique. for exampie,
`
`Ens teachnioue described in the aforementioned nonpatent iitorsturo ii.
`
`@0513
`
`Subsequently. presumption: of two vector ow which specifies the direction and passing position of the
`aforementioned axis line Lw in conversion Trauma {theta nw, ow) when accompanied by the above»
`mentioned rotation, respectiveiy, and ow is performed, Since this vector nw and ow are the parameters
`which oetermino the aforemohtioned axis iino Lw, flexibility is set to 4. Then. if in charge of those
`presumption tho throw-dimensionai oiace marker which moves a centering on aforementioned axis
`iine Lw circular orbit too to the insert oortion 32 of the aforementioned straibism endoscope ii}
`aooording to rotation of this insert portion i2 is provided first. The aforementioned vector Cw
`oorrssponds to the center of the cirouiar orbit of this throefiimonsionsi piaoo marker, it corresponds
`to the normai iino direction of a plans including this oirouiar orbit, and it becomes the aforementionod
`vector ow to deai with the vector which passes aiong the center of this oirouiar orbit. The position
`sensor for detecting the: position in the oforemsntionetl worio coordinate system So; of this camera
`part ‘i
`‘i
`is provided aiso to the aforementioned camera part H of the aforementioned strabism
`EndOSjoooo i0. and the: position of the: camera part ii is measured to it.
`0052
`Next. if the aforementioned insert portion 132 is rotated the procedure oi’ measuring the position of
`the aforementioned throoodimonsio‘nai piaoo marker is repeated two or more times and is performed.
`if the position moasuromont vaiue of a threehdimsnsionai place marker is made into o’w ii: i, 2,
`n),
`nw which fiiis a iowor type (9) wiii be caicuiatoci by iinoarity presumption.
`{0053]
`{Mathematioai iormuia 3}
`
`min 2511“er iiiyli‘i‘ (i i? i z) ‘
`
`1
`
`. i. {$3}
`
`{@0541
`here ~~ {3“} expresses on inner product and **** expresses a Euciidean norm?
`{0055]
`Subsequently, Cw which fiils a iowor tyoe {in} is caicuiated from this oalouiated ow using noniinear
`presumption for example, the LevonborgMii/iarouadt method,
`{0058]
`‘
`{Mathomatioai formuia 4}
`
`1"
`min ‘1 ii
`
`x
`
`.
`'
`.
`2
`.i
`. o 3
`imam I git“ Cw i + {new mix“ Cwii
`)
`
`i
`
`‘ v
`
`Q
`
`,-
`\ 3‘ G 73
`
`£00513]
`
`

`

`ii’w «200-414 :393‘3
`
`{MES
`
`igoro 3*" (3*) expresses an inner product and **** expresses a Euoiioeen norm:
`Then, the aforementioned angie of rotation theta is searched for by a iower type {i i) from the angle
`which caiouiated pin-ow makes.
`{0059]
`,
`{Mathematioai formuia 5}
`
`it)w 15“.: $1:va Cori ]
`
`i} ~=ztrcmn WWW _
`i
`~
`{1:3:»w in: 3 pix-"Aw (“vi-X)
`
`ee;
`
`M‘x
`
`11.
`
`'54,!
`
`{6030]
`here w {w} expresses an inner product {and} exprees an outer product, and **** exoreeeee e
`Euciidoan norm.
`£00813]
`Vector no which expresses the direction of the aforementioned optimexis Lg to the East in the
`camera ooordinato system in the ebovermentionedfi conversion Time i“ theta, no on} of spin
`compensation. Vector co which expresses the position of the shoot of the aforementioned optiomaxie
`Lo in a camera coordinate system is presumed in these presumption, it corresponds to each
`aforementioned vector co. no, and the angie of rotation theta first. Ceiouietive image projection point
`10 (p’woiaj, thetam o. co} oi point p‘wmm on a world coordinate syotem is oaiouieted using an upper type
`{3} and {4t Next, each aforementioned vector cc and m: which iii! a flower typo {iii} are caiouiated by
`making the position, vector on the aotuai imagomooordinatee system of o‘wm into Sci
`{0052]
`{Mathematioai formuia i3}
`
`.
`f .
`q,
`'
`_
`‘
`i?
`311211231icimwjiitinwmi
`
`‘
`-
`w
`‘
`H m}
`
`{0083]
`< Simuietion experiment)
`Next, it describes about the simoiatioo experiment conducted about presumption of vector co
`mentioned above and no. The generated data for a simuietion is as toiiowe That is. the pattern of 4
`mm of intervais io every direction and the sequence of points by which the point was arranged 3x3
`was used for the oaiibretion as an observing object. The gauee noise with a standard deviation of 8.1
`mm was added to each point, And the insert portion of the etrebiem endoscope was rotated by a unit
`of about 45 degree, and the figure of the projected point of each ooint of the aforementioned pattern
`by the projection modei mentioned above was created by aesumption ofimaging the aforementioned
`Q
`pattern arrangedin the position which is separated from every:four sheets {tour frames} i and on
`
`observation port about 30—48 cm for every rotation ‘2‘“ W59 The range of an image face {imaging
`Surface} was aeeumed to be the range of 213%400 pixeis ipixei)
`{0064]
`On an image face. the simuiation roeuit which added the standard deviation (3.5. it). and a 3.5-“pixei
`goose noise, resoeo‘tiveiy is shown in Tobie i. The error of the remainder of a projected point and
`vector co, and no is a difference of the true veioe and point estimate which were given in the
`simoiotion. in orderto clarify reiation between the standard deviation of the given games. noieei end
`