In This paper we present an analytical approach to evaluate the performance analysis of SNR V/s BER of Optical System using MLSE Equalizer in fiber channel through the BPSK, QPSK, 64QAM &16QAM Modulation Technique. Finally we are comparing BER performance in Optical system using MLSE Equalizer through the modulation technique.