UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`Address: COMMISSIONER FOR PATENTS
`PO. Box 1450
`Alexandria, Virginia 2231371450
`www.uspto.gov
`
`15/801,082
`
`11/01/2017
`
`Mitsuo ABE
`
`PANDP0253US
`
`1034
`
`MARK D. SARALINO (PAN)
`RENNER, OTTO, BOISSELLE & SKLAR, LLP
`1621 EUCLID AVENUE
`19TH FLOOR
`CLEVELAND, OH 441 15
`
`LAM HUNG H
`
`ART UNIT
`2696
`
`PAPER NUMBER
`
`NOTIFICATION DATE
`
`DELIVERY MODE
`
`04/04/2019
`
`ELECTRONIC
`
`Please find below and/or attached an Office communication concerning this application or proceeding.
`
`The time period for reply, if any, is set in the attached communication.
`
`Notice of the Office communication was sent electronically on above—indicated "Notification Date" to the
`
`following e—mail address(es):
`
`ipdoeket@rennerotto.eom
`
`PTOL-90A (Rev. 04/07)
`
`

`

`0,7709 A0170” Summary
`
`Application No.
`15/801,082
`Examiner
`HUNG H LAM
`
`Applicant(s)
`ABE, Mitsuo
`Art Unit
`2696
`
`AIA (FITF) Status
`Yes
`
`- The MAILING DA TE of this communication appears on the cover sheet wit/7 the correspondence address -
`Period for Reply
`
`A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE g MONTHS FROM THE MAILING
`DATE OF THIS COMMUNICATION.
`Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed after SIX (6) MONTHS from the mailing
`date of this communication.
`|f NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication.
`-
`- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133).
`Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any earned patent term
`adjustment. See 37 CFR 1.704(b).
`
`Status
`
`1). Responsive to communication(s) filed on 01/31/19.
`[:1 A declaration(s)/affidavit(s) under 37 CFR 1.130(b) was/were filed on
`
`2a)D This action is FINAL.
`
`2b)
`
`This action is non-final.
`
`3). An election was made by the applicant in response to a restriction requirement set forth during the interview on 3
`1 December 2018; the restriction requirement and election have been incorporated into this action.
`
`4):] Since this application is in condition for allowance except for formal matters, prosecution as to the merits is
`closed in accordance with the practice under Expat/7e Quay/e, 1935 CD. 11, 453 O.G. 213.
`
`Disposition of Claims*
`
`5)
`
`Claim(s) fl is/are pending in the application.
`
`5a) Of the above claim(s)
`
`is/are withdrawn from consideration.
`
`Claim(s) 9 is/are allowed.
`
`Claim(s) 1—2 and 8 is/are rejected.
`
`Claim(s) Si is/are objected to.
`
`) ) ) )
`
`6 7
`
`8
`
`
`
`are subject to restriction and/or election requirement
`El Claim(s)
`9
`* If any claims have been determined aflowabie. you may be eligible to benefit from the Patent Prosecution Highway program at a
`
`participating intellectual property office for the corresponding application. For more information, please see
`
`http://www.jjgptggQV/patents/init_event§/pph/index.'§p or send an inquiry to PPeredhack@g§ptg.ggv.
`
`Application Papers
`10)E] The specification is objected to by the Examiner.
`
`11). The drawing(s) filed on 11/01/17 is/are: a). accepted or b)[:] objected to by the Examiner.
`Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a).
`
`Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121 (d).
`
`Priority under 35 U.S.C. § 119
`12). Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f).
`Certified copies:
`
`a). All
`
`b)C] Some**
`
`c)l:] None of the:
`
`1.. Certified copies of the priority documents have been received.
`
`2D Certified copies of the priority documents have been received in Application No.
`
`3D Copies of the certified copies of the priority documents have been received in this National Stage
`application from the International Bureau (PCT Rule 17.2(a)).
`
`** See the attached detailed Office action for a list of the certified copies not received.
`
`Attachment(s)
`
`1)
`
`Notice of References Cited (PTO-892)
`
`Information Disclosure Statement(s) (PTO/SB/08a and/or PTO/SB/08b)
`2)
`Paper No(s)/Mail Date_
`U.S. Patent and Trademark Office
`
`3) C] Interview Summary (PTO-413)
`Paper No(s)/Mail Date
`4) CI Other-
`
`PTOL-326 (Rev. 11-13)
`
`Office Action Summary
`
`Part of Paper No./Mai| Date 20190319
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 2
`
`DETAILED ACTION
`
`Notice of Pre-AIA or AIA Status
`
`1.
`
`The present application, filed on or after March 16, 2013, is being examined
`
`under the first inventor to file provisions of the AIA.
`
`Election/Restrictions
`
`2.
`
`Applicant's election with traverse of the species election requirement in the reply
`
`filed on 01/31/19 is acknowledged. Applicant’s argument is found persuasive.
`
`Therefore, the species election requirement mailed on 12/31/18 is hereby withdrawn.
`
`Claim Rejections - 35 USC § 102
`
`3.
`
`The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that
`
`form the basis for the rejections under this section made in this Office action:
`
`A person shall be entitled to a patent unless —
`
`(a)(1) the claimed invention was patented, described in a printed publication, or in public use,
`on sale or othenNise available to the public before the effective filing date of the claimed
`invention.
`
`(a)(2) the claimed invention was described in a patent issued under section 151, or in an
`application for patent published or deemed published under section 122(b), in which the
`patent or application, as the case may be, names another inventor and was effectively filed
`before the effective filing date of the claimed invention.
`
`4.
`
`Claim 1
`
`is rejected under 35 U.S.C. 102(a)(2)/(1) as being anticipated by Tokui
`
`(U82016/0142618).
`
`Regarding claim 1, Tokui discloses an imaging apparatus comprising:
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 3
`
`an imaging unit (Fig. 1: Imaging element 101) that captures a subject image
`
`while changing an in-focus position to generate a plurality of pieces of image data
`
`(abstract: [0034-0037]);
`
`an image processor (Fig. 1: 102/103) that synthesizes the plurality of pieces of
`
`image data generated by the imaging unit to generate still image data having a deeper
`
`depth of field than a depth of field of each of the plurality of pieces of image data ([0048;
`
`0062]: image information of different focus positons thus combine/composite to
`
`generate an extended depth of field image); and
`
`a controller (103) that controls the image processor (102), wherein the controller
`
`causes the image processor to detect a main subject from an image indicated by one
`
`image data in the plurality of pieces of image data ([0003; 0033; 0042]: subject A is
`
`designated as the focus position by face detection or user's designation),
`
`determine a range for synthesizing the plurality of pieces of image data based on
`
`a position of the detected main subject ([0042-0047]: focus position of image
`
`information C and B is set based on focus position of image A accordingly so that depth
`
`of field or depth rang of image information A-C are adjacent to each other respectively),
`
`and synthesize pieces of image data focused within the determined range in the
`
`plurality of pieces of image data to generate the still image data ([0048; 0062]: plurality
`
`of pieces of image information of different focus position within the determine
`
`adjacent/overlaps field of view or depth range in Figs. 2, 4, 6,7,8 are combined to
`
`generate an image with extended depth of field).
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 4
`
`Claim Rejections - 35 USC § 103
`
`5.
`
`In the event the determination of the status of the application as subject to AIA 35
`
`U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any
`
`correction of the statutory basis for the rejection will not be considered a new ground of
`
`rejection if the prior art relied upon, and the rationale supporting the rejection, would be
`
`the same under either status.
`
`6.
`
`The following is a quotation of 35 U.S.C. 103 which forms the basis for all
`
`obviousness rejections set forth in this Office action:
`
`A patent for a claimed invention may not be obtained, notwithstanding that the claimed
`invention is not identically disclosed as set forth in section 102, if the differences between the
`claimed invention and the prior art are such that the claimed invention as a whole would have
`been obvious before the effective filing date of the claimed invention to a person having
`ordinary skill in the art to which the claimed invention pertains. Patentability shall not be
`negated by the manner in which the invention was made.
`
`7.
`
`Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Tokui in
`
`View of Kojima (U82011/0181911).
`
`Regarding claim 2, Tokui fails to disclose: ““wherein the controller calculates
`
`importance of each of a plurality of subjects included in the image, and sets the main
`
`subject from the plurality of subjects based on the calculated importance”.
`
`In an analogous of art, Kojima discloses a camera system and face important
`
`degree detection section (56) and for finding important degree of face regions of the
`
`main images and setting a common display position of a sub image with respect to a
`
`main image based on the importance degree of a face region in each of the main
`
`images (abstract; Figs. 2-3: 56; 8180-190).
`
`In light of the teaching from Kojima, it would
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 5
`
`have been obvious to one of ordinary skill in the art before the effective filing date to
`
`modify the camera system of Tokui to include the face important degree detection
`
`section of Kojima. The modifications thus provide a means setting a common display
`
`position of a sub image with respect to a main image based on the importance degree
`
`of a face regions in each of the main images (Kojima: abstract; Figs. 2-3: 56; 8180-190)
`
`8.
`
`Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Tokui in
`
`view of Ramrattan (U82011/0286639).
`
`Regarding claim 8, Tokui fails to explicitly disclose the imaging apparatus
`
`according to claim 1, wherein the imaging unit captures the subject image to generate
`
`moving image data, and the plurality of pieces of image data is a plurality of pieces of
`
`frame image data constituting the moving image data.
`
`In an analogous of art, Ramrattan discloses a camera system for combining
`
`images to form an extended depth of field, capturing self-portraits, finger print and video
`
`([0039; 0046]: video and moving image data are inherently include a plurality of image
`
`data or a plurality of pieces of frame image data in order to provide video or moving
`
`image effects) .
`
`In light of the teaching from Ramrattan, it would have been obvious to
`
`one of ordinary skill in the art before the effective filing date to modify the camera
`
`system of Tokui to include the camera system of Ramrattan. The modifications thus
`
`provide a more versatile camera system that is capable of capturing self-portrait, finger
`
`print and video.
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 6
`
`9.
`
`Claim 9 is allowed.
`
`Allowable Subject Matter
`
`10.
`
`Claims 3-4 and 7 are objected to as being dependent upon a rejected base claim,
`
`but would be allowable if rewritten in independent form including all of the limitations of
`
`the base claim and any intervening claims.
`
`11.
`
`The following is an examiner’s statement of reasons for allowance:
`
`Regarding claim 3, the prior art of Tokui discloses a camera for combining
`
`images to create extended depth of field. The prior art of Ramrattan discloses a camera
`
`for capturing video/self—portrait and fingerprint and combining images to create
`
`extended depth of field. The prior art of Ajito (U82017/0223250) discloses a camera to
`
`generate image with larger depth of field. The prior art of Kinnoshita (U82014/0334681)
`
`discloses an image processing apparatus including a candidate detection unit
`
`configured to detect each of candidate images serving as candidates for a main subject
`
`for a plurality of frames of image data, and a main subject determination unit configured
`
`to obtain a degree of stable presence of the candidate images detected by the
`
`candidate detection unit within the image data spanning the plurality of frames and to
`
`determine a main subject among the candidate images using the degree of stable
`
`presence. Thus, while many references teaches camera systems for extending depth of
`
`field and determining candidate for main subject, none of the references alone or in
`
`combination, provide a motivation to teach the imaging apparatus according to claim 2
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 7
`
`further in combination with: “wherein the importance is set to a larger value as an area
`
`of each of the plurality of subjects in the image is larger”.
`
`Regarding claim 4, the prior art of Tokui discloses a camera for combining
`
`images to create extended depth of field. The prior art of Ramrattan discloses a camera
`
`for capturing video/self-portrait and fingerprint and combining images to create
`
`extended depth of field. The prior art of Ajito (U82017/0223250) discloses a camera to
`
`generate image with larger depth of field. The prior art of Kinnoshita (U82014/0334681)
`
`discloses an image processing apparatus including a candidate detection unit
`
`configured to detect each of candidate images serving as candidates for a main subject
`
`for a plurality of frames of image data, and a main subject determination unit configured
`
`to obtain a degree of stable presence of the candidate images detected by the
`
`candidate detection unit within the image data spanning the plurality of frames and to
`
`determine a main subject among the candidate images using the degree of stable
`
`presence. Thus, while many references teaches camera systems for extending depth of
`
`field and determining candidate for main subject, none of the references alone or in
`
`combination, provide a motivation to teach the imaging apparatus according to claim 2
`
`further in combination with: “wherein the importance is set to a larger value as a
`
`position of each of the plurality of subjects is closer to a center of the image”.
`
`Regarding claim 7, the prior art of Tokui discloses a camera for combining
`
`images to create extended depth of field. The prior art of Ramrattan discloses a camera
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 8
`
`for capturing video/self-portrait and fingerprint and combining images to create
`
`extended depth of field. The prior art of Ajito (U82017/0223250) discloses a camera to
`
`generate image with larger depth of field. The prior art of Kinnoshita (U82014/0334681)
`
`discloses an image processing apparatus including a candidate detection unit
`
`configured to detect each of candidate images serving as candidates for a main subject
`
`for a plurality of frames of image data, and a main subject determination unit configured
`
`to obtain a degree of stable presence of the candidate images detected by the
`
`candidate detection unit within the image data spanning the plurality of frames and to
`
`determine a main subject among the candidate images using the degree of stable
`
`presence. Thus, while many references teaches camera systems for extending depth of
`
`field and determining candidate for main subject, none of the references alone or in
`
`combination, provide a motivation to teach the imaging apparatus according to claim 1
`
`further in combination with: “wherein, when causing the image processor to detect the
`
`main subject, the controller causes the image processor to use information in which
`
`information for identifying each of the plurality of pieces of image data, information
`
`indicating a focused region in each of the plurality of pieces of image data, and the in-
`
`focus position in generating each of the plurality of pieces of image data are associated
`
`with each other”.
`
`Regarding independent claim 9, the prior art of Tokui discloses a camera for
`
`combining images to create extended depth of field. The prior art of Ramrattan
`
`discloses a camera for capturing video/self-portrait and fingerprint and combining
`
`images to create extended depth of field. The prior art of Ajito (US2017/0223250)
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 9
`
`discloses a camera to generate image with larger depth of field. The prior art of
`
`Kinnoshita (US2014/0334681) discloses an image processing apparatus including a
`
`candidate detection unit configured to detect each of candidate images serving as
`
`candidates for a main subject for a plurality of frames of image data, and a main subject
`
`determination unit configured to obtain a degree of stable presence of the candidate
`
`images detected by the candidate detection unit within the image data spanning the
`
`plurality of frames and to determine a main subject among the candidate images using
`
`the degree of stable presence. Thus, while many references teaches camera systems
`
`for extending depth of field and determining candidate for main subject, none of the
`
`references alone or in combination, provide a motivation to teach: “wherein in a case
`
`where a first subject and a second subject are disposed in each image of the plurality of
`
`pieces of image data, when areas of the first subject and the second subject in the
`
`image are different from each other, in-focus positions of the first subject and the
`
`second subject are different from each other, and center positions of the first subject
`
`and the second subject in the image are equal to each other, one of the first subject and
`
`the second subject having a larger area in the image is set as the main subject, and
`
`when the areas of the first subject and the second subject in the image are equal to
`
`each other, the in-focus positions of the first subject and the second subject are different
`
`from each other, and the center positions of the first subject and the second subject in
`
`the image are different from each other, one of the first subject and the second subject
`
`closer to a center of the image is set as the main subject", in combination with all other
`
`limitations of the claim.
`
`

`

`Application/Control Number: 15/801 ,082
`Art Unit: 2696
`
`Page 10
`
`Regarding dependent claims 5 and 6 the claims are allowed as being dependent
`
`of claim 2.
`
`Conclusion
`
`12.
`
`Any inquiry concerning this communication or earlier communications from the
`
`examiner should be directed to HUNG H LAM whose telephone number is (571)272-
`
`7367. The examiner can normally be reached on 9AM-5PM.
`
`Examiner interviews are available via telephone, in-person, and video
`
`conferencing using a USPTO supplied web-based collaboration tool. To schedule an
`
`interview, applicant is encouraged to use the USPTO Automated Interview Request
`
`(AIR) at http://www.uspto.gov/interviewpractice.
`
`If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
`
`supervisor, SINH TRAN can be reached on (571) 272-7564. The fax phone number for
`
`the organization where this application or proceeding is assigned is 571-273-8300.
`
`Information regarding the status of an application may be obtained from the
`
`Patent Application Information Retrieval (PAIR) system. Status information for
`
`published applications may be obtained from either Private PAIR or Public PAIR.
`
`Status information for unpublished applications is available through Private PAIR only.
`
`For more information about the PAIR system, see http://pair-direct.uspto.gov. Should
`
`you have questions on access to the Private PAIR system, contact the Electronic
`
`Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a
`
`USPTO Customer Service Representative or access to the automated information
`
`system, call 800-786-9199 (IN USA OR CANADA) or 571-272—1000.
`
`