UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`Address: COMMISSIONER FOR PATENTS
`PO. Box 1450
`Alexandria, Virginia 2231371450
`www.uspto.gov
`
`14/ 199,203
`
`03/06/2014
`
`Hiroshi SAITO
`
`AOYAP0150US
`
`9095
`
`MARK D. SARALINO (PAN)
`RENNER, OTTO, BOISSELLE & SKLAR, LLP
`1621 EUCLID AVENUE
`19TH FLOOR
`CLEVELAND, OH 441 15
`
`DEMOSKY~ PATRICK E
`
`ART UNIT
`2486
`
`PAPER NUMBER
`
`NOTIFICATION DATE
`
`DELIVERY MODE
`
`09/17/2018
`
`ELECTRONIC
`
`Please find below and/or attached an Office communication concerning this application or proceeding.
`
`The time period for reply, if any, is set in the attached communication.
`
`Notice of the Office communication was sent electronically on above—indicated "Notification Date" to the
`
`following e—mail address(es):
`
`ipdoeket@rennerotto.eom
`
`PTOL-90A (Rev. 04/07)
`
`

`

`Off/09 A0170” Summary
`
`Application No.
`14/199,203
`Examiner
`PATRICK E DEMOSKY
`
`Applicant(s)
`SAITO et al.
`Art Unit
`2486
`
`AIA Status
`Yes
`
`- The MAILING DA TE of this communication appears on the cover sheet wit/7 the correspondence address -
`Period for Reply
`
`A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE g MONTHS FROM THE MAILING
`DATE OF THIS COMMUNICATION.
`Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed
`after SIX (6) MONTHS from the mailing date of this communication.
`|f NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication.
`-
`- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133).
`Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any
`earned patent term adjustment. See 37 CFR 1.704(b).
`
`Status
`
`1). Responsive to communication(s) filed on 3/27/2018.
`[:1 A declaration(s)/affidavit(s) under 37 CFR 1.130(b) was/were filed on
`
`2a)D This action is FINAL.
`
`2b)
`
`This action is non-final.
`
`3)[:] An election was made by the applicant in response to a restriction requirement set forth during the interview on
`; the restriction requirement and election have been incorporated into this action.
`
`4)[:] Since this application is in condition for allowance except for formal matters, prosecution as to the merits is
`closed in accordance with the practice under Expat/7e Quay/e, 1935 CD. 11, 453 O.G. 213.
`
`Disposition of Claims*
`
`5)
`
`Claim(s) fl is/are pending in the application.
`
`5a) Of the above claim(s)
`
`is/are withdrawn from consideration.
`
`E] Claim(s)
`
`is/are allowed.
`
`Claim(s) fl is/are rejected.
`
`[:1 Claim(s) _ is/are objected to.
`
`) ) ) )
`
`6 7
`
`8
`
`
`
`are subject to restriction and/or election requirement
`[j Claim(s)
`9
`* If any claims have been determined aflowabte. you may be eligible to benefit from the Patent Prosecution Highway program at a
`
`participating intellectual property office for the corresponding application. For more information, please see
`
`http://www.uspto.gov/patents/init events/pph/index.jsp or send an inquiry to PPeredback@uspto.gov.
`
`Application Papers
`10)[:] The specification is objected to by the Examiner.
`
`11)[:] The drawing(s) filed on
`
`is/are: a)D accepted or b)l:] objected to by the Examiner.
`
`Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a).
`Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121 (d).
`
`Priority under 35 U.S.C. § 119
`12). Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f).
`Certified copies:
`
`a). All
`
`b)[:l Some**
`
`c)C] None of the:
`
`1..
`
`Certified copies of the priority documents have been received.
`
`2.[:]
`
`Certified copies of the priority documents have been received in Application No.
`
`3.[:] Copies of the certified copies of the priority documents have been received in this National Stage
`application from the International Bureau (PCT Rule 17.2(a)).
`
`** See the attached detailed Office action for a list of the certified copies not received.
`
`Attachment(s)
`
`1)
`
`Notice of References Cited (PTO-892)
`
`2) E] Information Disclosure Statement(s) (PTO/SB/08a and/or PTO/SB/08b)
`Paper No(s)/Mail Date_
`U.S. Patent and Trademark Office
`
`3) C] Interview Summary (PTO-413)
`Paper No(s)/Mail Date
`4) D Other-
`
`PTOL-326 (Rev. 11-13)
`
`Office Action Summary
`
`Part of Paper No./Mai| Date 20180831
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 2
`
`DETAILED ACTION
`
`Notice of Pre-AIA or AIA Status
`
`1.
`
`The present application, filed on or after March 16, 2013, is being examined under the first
`
`inventor to file provisions of the AIA.
`
`Priority
`
`2.
`
`Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
`
`The certified copy has been filed in parent Application No. JP2013-078294, filed on 4/4/2013.
`
`Information Disclosure Statement
`
`3.
`
`The information disclosure statement submitted on 3/6/2014 is in compliance with the provisions
`
`of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
`
`Response to Arguments
`
`4.
`
`Applicant's arguments filed 3/27/2018 have been fully considered but they are directed towards
`
`newly amended claim language.
`
`See the rejection below for how the cited art in light of new/existing references reads on the
`
`newly amended language as well as the examiner’s interpretation of the cited art in view of the presented
`
`claim set.
`
`Claim Rejections - 35 USC § 103
`
`5.
`
`The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
`
`rejections set forth in this Office action:
`
`A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not
`identically disclosed as set forth in section 102, if the differences between the claimed invention and the
`prior art are such that the claimed invention as a whole would have been obvious before the effective
`filing date of the claimed invention to a person having ordinary skill in the art to which the claimed
`invention pertains. Patentability shall not be negated by the manner in which the invention was made.
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 3
`
`6.
`
`The factual inquiries set forth in Graham v. John Deere C0., 383 US. 1, 148 USPQ 459 (1966),
`
`that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are
`
`summarized as follows:
`
`1. Determining the scope and contents of the prior art.
`
`2. Ascertaining the differences between the prior art and the claims at issue.
`
`3. Resolving the level of ordinary skill in the pertinent art.
`
`4. Considering objective evidence present in the application indicating obviousness or
`
`nonobviousness.
`
`7.
`
`Claims 1, 2, and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Foote (US
`
`20020122113 A1) (hereinafter Foote) in view of itself in view of Monroe (US 7023913 B 1) (hereinafter
`
`Monroe).
`
`Regarding claim 1, Foote discloses:
`
`An imaging system for shooting a plurality of images to generate a panoramic image of a
`
`subject region, comprising: [See Foote, (fl 0015, 0022 discloses using a camera array to capture
`
`plural piecewise continuous images of a scene, thus forming a panoramic image]
`
`a plurality of cameras arranged in a two-dimensional array, wherein [See Foote, Fig. 1B
`
`illustrates a two-dimensional array of cameras.]
`
`each camera of the plurality of cameras shoots an image of a corresponding
`
`shooting region
`
`the subject region is divided along a first direction into the respective shooting
`
`regions of the plurality of cameras, [See Foote, Fig. 2A illustrates an example With
`
`“camera 1 View” (210), “camera 2 View” (220), “camera 3 View” (230), in which a
`
`scene is segmented into three regions along a horizontal/landscape panoramic
`
`image.]
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 4
`
`the shooting regions of the plurality of cameras are arranged adjacent to one
`
`another along the first direction, such that a combination of the images of the shooting
`
`regions shot by the plurality of cameras forms the panoramic image of the subject region,
`
`[See Foote, ‘J[ 0061 discloses abutting images from each of camera 1 View and camera
`
`2 View. It is additionally disclosed that the abutting areas are combined into a single
`
`panoramic image.]
`
`Foote does not appear to explicitly disclose:
`
`two cameras haVing neighboring shooting regions along the first direction
`
`form a camera couple;
`
`all of the two cameras of each camera couple are arranged in the array
`
`adjacent to one another in either the first direction or in a second direction orthogonal
`
`to the first direction, and
`
`the number of camera couples including two cameras that are arranged
`
`adjacent to one another in the first direction is less than the number of camera couples
`
`including two cameras that are arranged adjacent to one another in the second
`
`direction.
`
`However, Foote teaches:
`
`two cameras haVing neighboring shooting regions along the first direction form a camera
`
`couple; [See Foote, ‘JI 0058-0059 discloses that many configurations of cameras (particularly,
`
`cameras arranged in a planar array) are possible. Figs. 1A, 1B, and 1C illustrate examples
`
`of circular, planar array, and linear array configurations. Though Foote’s Fig. 1B
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 5
`
`illustrates a 5x3 (MXN) array of cameras, it is not intended for this illustration to be a
`
`limiting aspect of Foote’s disclosure, and is merely an example of one of many possible
`
`camera configurations; The Examiner notes as a preliminary matter that as currently
`
`claimed, "a camera couple" appears to be for the purposes of explaining the positional
`
`relationship between cameras in a planar array, as well as potentially including contextual
`
`relationship between cameras and their corresponding "shooting regions" in the subject
`
`
`area. While Foote does not explicitly disclose “camera couples”, Fig. 1B clearly illustrates a
`
`planar array of cameras oriented with respect to one another along first (horizontal) and
`
`second (vertical) directions. As further elucidated below, Foote’s disclosure discusses that
`
`each camera of would be capable of being aimed at a particular region of a scene. Figures
`
`14 and 15 further illustrate regions of scene as imaged by cameras, and ultimately being
`
`stitched together to form a panoramic image in a manner similar to the instant application.]
`
`the number of camera couples including two cameras that are arranged adjacent to one
`
`another in the first direction is less than the number of camera couples including two cameras that
`
`are arranged adjacent to one another in the second direction. [See Foote, ‘fl 0058-0059 discloses
`
`that many configurations of cameras (particularly, cameras arranged in a planar array) are
`
`possible. Figs. 1A, 1B, and 1C illustrate examples of circular, planar array, and linear
`
`array configurations. Though Foote’s Fig. 1B illustrates a 5x3 (MxN) array of cameras, it is
`
`not intended for this illustration to be a limiting aspect of Foote’s disclosure, and is merely
`
`an example of one of many possible camera configurations. It is thus understood by one of
`
`ordinary skill in the art that the selection of dimensions (MXN) of such a camera array
`
`arrangement is within the level of ordinary skill.
`
`It would have been obvious to the person having ordinary skill in the art to have modified
`
`the imaging system disclosed by Foote by adding the system capable of aiming each camera at a
`
`particular region of a scene as taught by Foote. One of ordinary skill in the art would have been
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 6
`
`motivated to make this modification in order to allow for capturing particular points of interest in
`
`a scene as well as panorama generation.
`
`Foote does not appear to explicitly disclose:
`
`all of the two cameras of each camera couple are arranged in the array
`
`adjacent to one another in either the first direction or in a second direction orthogonal
`
`to the first direction, and
`
`However, Monroe discloses:
`
`all of the two cameras of each camera couple are arranged in the array adjacent to one
`
`another in either the first direction or in a second direction orthogonal to the first direction, and
`
`[See Monroe, Fig. 4B illustrates cameras in an array configuration with horizontal angular
`
`offset, adapted for panoramic image capture; See Monroe, Fig. 36 illustrates cameras in an
`
`array configuration with vertical angular offset, adapted for panoramic image capture; See
`
`Monroe, Fig. 27 illustrates panning an array camera of any geometry in the X and y-axes;
`
`See Monroe, col. 11 line 64 — col. 12 line 32 notes that the various camera configurations are
`
`reconstructing an entire panoramic area being covered. Explicit note is made in this section
`
`to an implementation in which all of the zones are shown in a strip for a full panoramic
`
`View; See Monroe, Fig. 15 illustrates a strip View of a panorama produced by a camera
`
`configuration such as in Fig. 5. See Monroe, col. 16 lines 27-42 discloses details related to
`
`Figs. 33 and 35, which illustrates a stacked array camera configuration in which the
`
`cameras are vertically angularly offset so as to image a panoramic scene similarly. It is
`
`noted that the cameras are immediately adjacent]
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 7
`
`It would have been obvious to the person having ordinary skill in the art to modify the
`
`invention disclosed by Foote to add the teachings of Monroe in order to provide full field imaging
`
`of a panoramic scene by stacking or arranging camera sensors in an array.
`
`Regarding claim 2, Foote discloses all the limitations of claim 1, and is analyzed as
`
`previously discussed with respect to that claim.
`
`Foote discloses:
`
`wherein at least one of the camera couples includes two cameras handling shooting
`
`regions encompassing a part of the subject region which includes no end of the subject region in
`
`the first direction. [See Foote, Fig. 2A illustrates a panoramic scene comprised of several
`
`views from corresponding cameras. The View from camera 2, for instance, is shown as not
`
`including an "end” of the subject region. As earlier discussed, the particular camera
`
`arrangement being used can be chosen according to a variety of configurations. As such, it
`
`is understood that choosing shooting regions corresponding with a selected arrangement of
`
`cameras is Within the level of ordinary skill in the art.]
`
`(Examiner’s Note: This is further evidenced by Carlsson et al. (US 7864215 B2), which,
`
`in Figs. 1-4 and col. 5 lines 21-35 illustrates/discloses that it is conceivable for “different cameras
`
`to contribute to different parts of a wide image” according to a variety of camera arrangements —
`
`including arrangements in which the cameras are mounted adjacent to one another in a "vertica ”
`
`direction”)
`
`Regarding claim 7, Foote in view of Monroe discloses all the limitations of claim 1, and
`
`is analyzed as previously discussed with respect to that claim.
`
`Monroe discloses:
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 8
`
`wherein the subject region divided along the first direction is shot by the camera couples
`
`arranged in the array adjacent to one another in both the first direction and the second direction.
`
`[See Monroe, Fig. 4B illustrates cameras in an array configuration with horizontal angular
`
`offset, adapted for panoramic image capture; See Monroe, Fig. 36 illustrates cameras in an
`
`array configuration with vertical angular offset, adapted for panoramic image capture; See
`
`Monroe, Fig. 27 illustrates panning an array camera of any geometry in the X and y-axes;
`
`See Monroe, col. 11 line 64 — col. 12 line 32 notes that the various camera configurations are
`
`reconstructing an entire panoramic area being covered. Explicit note is made in this section
`
`to an implementation in which all of the zones are shown in a strip for a full panoramic
`
`view. Throughout Monroe’s disclosure as a whole, many embodiments are shown in which
`
`cameras are stacked horizontally and vertically in an array, and are angularly offset (for
`
`instance, as shown in Figs. 4B, 35, and 36). The angular offset of the stacked cameras thus
`
`allows for the array configuration as in Fig. 27 to be represented in the panoramic image
`
`strip as earlier mentioned.]
`
`8.
`
`Claims 3, 4, and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Foote in view of
`
`Monroe in view of Yasushi (JP 2011-176460 A) (hereinafter Yasushi).
`
`Regarding claim 3, Foote in view of Monroe discloses all the limitations of claim 1, and
`
`is analyzed as previously discussed with respect to that claim.
`
`Foote discloses:
`
`wherein when the shooting regions located continuously from one end of the subject
`
`region to another end of the subject region are sequentially allocated to the respective cameras,
`
`[See Foote, Fig. 2A illustrates an exemplary embodiment in which a wide image of a scene
`
`divided into views 210, 220, 230 obtained from cameras 1, 2, and 3, respectively; See Foote,
`
`‘JI 0060-0061 discloses that “abutting areas are combined into a single panoramic image."]
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 9
`
`Foote in view of Monroe does not explicitly disclose:
`
`the plurality of cameras are arranged so that a trace of positions of the
`
`cameras has a U-shape when the cameras are traced in the order of allocation of the
`
`shooting regions allocated to the cameras.
`
`However, Yasushi discloses:
`
`the plurality of cameras are arranged so that a trace of positions of the cameras has a U-
`
`shape when the cameras are traced in the order of allocation of the shooting regions allocated to
`
`the cameras. [See Yasushi, Fig. 2 illustrates a plurality of camera units are allocated
`
`clockwise as 1, 2, 4, 3; See Yasushi, Fig. 4 illustrates that tracing cameras in the allocated
`
`order 1a, 2a, 4a, 3a would create a U-shape. Such an arbitrary allocation positioning of
`
`cameras so as to trace a particular shape would be understood as being Within the level of
`
`ordinary skill in the art.]
`
`It would have been obvious to the person having ordinary skill in the art to modify the
`
`invention disclosed by Foote in view of Monroe to add the teachings of Yasushi in order to
`
`provide a control means for controlling the imaging direction of the first imaging unit to fourth
`
`imaging units, thus providing an articulated ability to vary the size and amount of overlap
`
`between imaging regions. (Yasushi, ‘fl 0005)
`
`Regarding claim 4, Foote in view of Monroe discloses all the limitations of claim 1, and
`
`is analyzed as previously discussed with respect to that claim.
`
`Foote discloses:
`
`wherein when the shooting regions located continuously from one end of the subject
`
`region to another end of the subject region are sequentially allocated to the respective cameras,
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 10
`
`[See Foote, Fig. 2A illustrates an exemplary embodiment in which a wide image of a scene
`
`divided into views 210, 220, 230 obtained from cameras 1, 2, and 3, respectively; See Foote,
`
`‘JI 0060-0061 discloses that “abutting areas are combined into a single panoramic image."]
`
`Yasushi discloses:
`
`the plurality of cameras are arranged so that a trace of positions of the cameras have a
`
`traversable shape when the cameras are traced in the order of allocation of the shooting regions to
`
`the cameras. [See Yasushi, Fig. 2 illustrates a plurality of camera units are allocated
`
`clockwise as 1, 2, 4, 3; See Yasushi, Fig. 4 illustrates that tracing cameras in the allocated
`
`order 1a, 2a, 4a, 3a would create a U-shape. Such an arbitrary allocation positioning of
`
`cameras so as to trace a particular shape would be understood as being within the level of
`
`ordinary skill in the art.]
`
`Regarding claim 5, Foote in View of Monroe discloses all the limitations of claim 1, and
`
`is analyzed as previously discussed with respect to that claim.
`
`Yasushi discloses:
`
`wherein the plurality of cameras include four cameras with two cameras arranged in a
`
`lateral direction and two cameras arranged in a vertical direction. [See Yasushi, Fig. 4 illustrates
`
`camera units 1 and 2 disposed horizontally and camera units 1 and 3 disposed vertically.]
`
`9.
`
`Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Foote in View of Monroe in
`
`View of Aman (US 20070279494 A1) (hereinafter Aman).
`
`Regarding claim 6, Foote in View of Monroe discloses all the limitations of claim 1, and
`
`is analyzed as previously discussed with respect to that claim.
`
`Foote in View of Monroe does not explicitly disclose:
`
`

`

`Application/Control Number: 14/199,203
`Art Unit: 2486
`
`Page 11
`
`wherein the plurality of cameras include nine cameras with three cameras
`
`arranged in a lateral direction and three cameras arranged in a vertical direction.
`
`However, Aman discloses:
`
`wherein the plurality of cameras include nine cameras with three cameras arranged in a
`
`lateral direction and three cameras arranged in a vertical direction. [See Aman, Fig. 3 illustrates
`
`nine cameras disposed in a 3X3 grid. Such an arbitrary allocation positioning of cameras
`
`would be understood as being within the level of ordinary skill in the art.]
`
`It would have been obvious to the person having ordinary skill in the art to modify the
`
`invention disclosed by Foote in view of Monroe to add the teachings of Aman in order to provide
`
`a larger grid of cameras so as to subsequently capture a larger panoramic image with greater
`
`coverage of a scene of interest.
`
`11.
`
`The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
`
`Conclusion
`
`LS 7864215 B2
`
`Carlsson; Stefan et al.
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`0
`
`LS 20070189747 A1
`
`L S 20100085422 A1
`
`LS 20060238617 A1
`
`L S 20100097444 A1
`
`L S 20120262607 A1
`
`L S 20070172151 A1
`
`LS 20130016181 A1
`
`L S 9204041 B1
`
`
`
`Ujisato; Takanobu et al.
`
`Yamashita; Noriyuki et al.
`
`Tamir; Michael
`
`Lablans; Peter
`
`Shimura; Tomoya et al.
`
`Gennetten et al.
`
`Penner
`
`Campbell
`
`LS 20130044181 A1
`
`Baker etal.
`
`