March 26, 2024

• Requested an Digital evalution version of Foundations of Computer Vision

February 21, 2024

• Waymo will also publish the 2024 Challenges in mid-March, which will run through May 2024. There will be four challenges: Motion Prediction, Sim Agents, 3D Semantic Segmentation and Occupancy and Flow prediction.
• The last challenge overlaps nicely with the OpenDrive lab challenge.

February 20, 2024

• Submitted two project proposals for CV2, based on Mapless Driving and Occupancy and Flow.
• The Mapless-driving challenge also pointed to LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving, a paper with code.
• Occupancy and flow challenge pointed to Fully Sparse 3D Panoptic Occupancy Prediction, a paper where the code will be made available.

February 19, 2024

• In April 2024 this new book will come out: Foundations of Computer Vision

February 15, 2024

• Checked the Robotic Vision Challenge slack-channel, but not much information updates since the beginning of 2023.
• Read Scene as Occupancy, paper with code.
• This paper was cited 22x, including these arXiv papers:
  • Occworld: Learning a 3d occupancy world model for autonomous driving, arXiv 2023
  • Selfocc: Self-supervised vision-based 3d occupancy prediction, arXiv 2023
  • OccNeRF: Self-Supervised Multi-Camera Occupancy Prediction with Neural Radiance Fields, arXiv 2023
  • RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation, arXiv 2023
  • WoVoGen: World volume-aware diffusion for controllable multi-camera driving scene generation, arXiv 2023
  • COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction, arXiv 2023
  • BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection, arXiv 2023
  • PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation, arXiv 2023
• This paper was also cited in IEEE journals:
  • 3DOPFormer: 3D Occupancy Perception from Multi-Camera Images with Directional and Distance Enhancement IEEE Transaction ... 2023
  • Multi-Camera Unified Pre-Training Via 3D Scene Reconstruction IEEE Robotics ... 2024. Paper with code. Note that comparisons are made on several datasets, the 3D occupancy prediction challenge is only Table VI (with only BEVStereo as algorithm mentioned)
• The 2023 Challenge is the 3D Occupancy prediction track, where the also point to Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (38x cited).
• The awards on the 2023 were given to:
  • FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation, paper (cited 12x) with code
  • OccTransformer: Improving BEVFormer for 3D camera-only occupancy prediction, no code link given
  • MiLO: Multi-task Learning with Localization Ambiguity Suppression for Occupancy Prediction, no code link given
• For CV2 the best description of last year's champion (Nvidia) would be their ICCV 2023 paper: FB-BEV: BEV Representation from Forward-Backward View Transformations (cited 9x, including COTR and BevNeXt).
• An interesting arXiv paper citing FB-BEV not listed above is OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries
• Also looked at the Multi-View 3D Visual Grounding challenge:
  
• Based on EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI, paper with code
• The last paper is not about driving scenes, but on indoor embodied 3D perception systems. For sure interesting for our group and @Home applications, but a bit of outlier of OpenDriveLab. In addition, it relies on language prompts, so a bit out of my expertise. Skip this challenge for the moment.

February 14, 2024

• Previous year 5 MscAI-students worked on the Robotic Vision Scene Understanding Challenge 2023
• No new call yet on the general challenge page.
• The BenchBot github hasn't been updated after the competition.
• So looking for another challenge.
• The Autonomous Grand Challenge from OpenDriveLab is in conjunction with CVPR 2024 (July, Seattle).
• There are three perception challenges. The DevKit will be released on March 01.
• The perception challenges are:
  • Occupancy and Flow
  • Multi-View 3D Visual Grounding
  • Mapless Driving
• Subscribed to the three slack-channels, but not much traffic there yet.
• For Occupancy and Flow the following literature is suggested:
  • Scene as Occupancy, paper with code.
• For Multi-View 3D Visual Grounding the following literature is suggested:
  • EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI, paper with code
• For Mapless Driving the following literature is suggested:
  • OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping, paper with code.
• The OpenLane benchmark is updated, on Nov 1, 2023 Dev2.1.0 was released. The test-server is still open. The last submission was 8 months ago. Three winners are MFV, qcraft2, Victory.
• MFV approach was based on Yolov8, with additional topological heads. It is cited in 4 papers:
  • Petrv2: A unified framework for 3d perception from multi-camera images (CVPR 2023, cited 157x).
  • High-Definition Maps Construction Based on Visual Sensor: A Comprehensive Survey (IEEE Transactions on Intelligent Vehicles, an overview paper which could be good starting point)
  • Stream Query Denoising for Vectorized HD Map Construction, (arXiv paper, Jan 2024)
  • Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps, (arXiv paper, Nov 2023)

Previous Labbooks

• Labbook 2023