(1) The public administrator and guardian or person appointed under KRS 395.390 shall, by order of the court, be the administrator or administrator de bonis non, or if there is a will, administrator with the will annexed, and shall have all the rights and powers, be subjected to the same liabilities, and be governed by the same laws prescribed for administrators and guardians. (2) The court may, however, at any time, set aside the order entrusting the estate to the public administrator and guardian or person appointed under KRS 395.390 and allow an executor or administrator to qualify.