With the rapidly increasing research interest in autonomous vehicles, map update systems have become crucial. In the existing method, the original map is compared with the sensor data, and the newly changed map data is updated unconditionally. However, this is a simple iterative updating method that cannot distinguishing the landmarks (e.g., building, crosswalk, etc.). In this study, objects (i.e., people, cars, etc.) that are not related to the map are detected using the deep learning technique. The objects are excluded from the 3D data using a camera and LidarDAR sensor fusion. The remaining undetected 3D data is compared with the map data, and the map is updated by adding new landmarks and simultaneously removing the missing landmarks. The location accuracy is increased by localization based on the updated map. Compared to the original map, this proposed deep-learning-based method can reduces the error by up to 1.5 m. Thus, this proposed method is expected to aid in the advancement of the map update system.