Total capacity exceeds 1 EB! How do you connect two HDFS clusters with different histories? Challenges and design decisions in data platform integration

LY Corporation described how it is integrating two massive HDFS platforms—one from former LINE and one from former Yahoo Japan—while reconciling different operating models, permission schemes, and namespace designs.

With more than 1EB of HDFS data under management, the company highlighted NameNode bottlenecks and the scaling limits that come with operating at exabyte scale.

The presentation also focused on how to design data linkage between platforms after organizational integration, balancing access control, governance, and operational flexibility.

The case offers practical guidance for large-scale Hadoop/HDFS operations and for connecting data platforms across merged organizations.