An Investigation of Cross-Phase Modulation and Machine Learning Based Alleviation of Nonlinear Effects in a DP-QPSK Based DWDM System
Corresponding author: M. N. Mahalakshmi
Abstract
Dense Wavelength Division Multiplexing (DWDM) systems with Dual-Polarization Quadrature Phase-Shift Keying (DP-QPSK) technology are enabling current optical fiber communication systems in Metropolitan Area Networks and Long-haul networks. The impact of the nonlinear impairments caused by the Kerr effect is more detrimental to the performance of DWDM systems. Among these impairments, the dominant impairment is Cross-Phase Modulation (XPM). In this paper, a nine-channel coherent DP-QPSK DWDM system with Standard Single Mode Fiber (SSMF) at a bit rate of 112 Gb/s per channel is simulated in OptiSystem v18. The study examines how XPM affects performance, specifically focusing on polarization crosstalk and phase noise for different channel spacings and transmission distances up to 900 km. The alleviation of nonlinear effects is demonstrated using two Machine Learning (ML) models: Bidirectional Long Short-Term Memory (Bi-LSTM) and Transformer. The efficacy of these machine learning techniques in mitigating nonlinear effects is evaluated against Digital Back Propagation (DBP). The findings demonstrate that the Transformer model outperforms both DBP and Bi-LSTM.
Keywords:
Cross-Phase Modulation, DP-QPSK, DWDM, nonlinear effectsDownloads
References
L. Li et al., "Nonlinear Polarization Crosstalk Canceller for Dual-Polarization Digital Coherent Receivers," in Optical Fiber Communication Conference (2010), paper OWE3, Mar. 2010, Art no. OWE3. DOI: https://doi.org/10.1364/OFC.2010.OWE3
Z. Tao et al., "Simple Fiber Model for Determination of XPM Effects," Journal of Lightwave Technology, vol. 29, no. 7, pp. 974–986, Apr. 2011. DOI: https://doi.org/10.1109/JLT.2011.2107728
E. Morsy, H. A. Fayed, A. Abd El Aziz, and M. H. Aly, "SPM and XPM crosstalk in WDM systems with DRA: Channel spacing and attenuation effects," Optics Communications, vol. 417, pp. 79–82, June 2018. DOI: https://doi.org/10.1016/j.optcom.2018.02.028
J. Ding et al., "Influence of equalization enhanced phase noise on digital nonlinearity compensation in Nyquist-spaced superchannel transmission systems," in Semiconductor Lasers and Applications XI, Oct. 2021, vol. 11891, pp. 38–45. DOI: https://doi.org/10.1117/12.2600923
C. Li et al., "Inter-Channel Fiber Nonlinearity Mitigation in High Baud-Rate Optical Communication Systems," Journal of Lightwave Technology, vol. 39, no. 6, pp. 1653–1661, Mar. 2021. DOI: https://doi.org/10.1109/JLT.2020.3042671
H. Rabbani, H. Hosseinianfar, H. Rabbani, and M. Brandt-Pearce, "Analysis of Nonlinear Fiber Kerr Effects for Arbitrary Modulation Formats," Journal of Lightwave Technology, vol. 41, no. 1, pp. 96–104, Jan. 2023. DOI: https://doi.org/10.1109/JLT.2022.3213182
Z. Tao, W. Yan, S. Oda, T. Hoshida, and J. C. Rasmussen, "A simplified model for nonlinear cross-phase modulation in hybrid optical coherent system," Optics Express, vol. 17, no. 16, pp. 13860–13868, Aug. 2009. DOI: https://doi.org/10.1364/OE.17.013860
M. Winter, C.-A. Bunge, D. Setti, and K. Petermann, "A Statistical Treatment of Cross-Polarization Modulation in DWDM Systems," Journal of Lightwave Technology, vol. 27, no. 17, pp. 3739–3751, Sept. 2009. DOI: https://doi.org/10.1109/JLT.2009.2025394
Y.-T. Hsueh et al., "Passband Narrowing and Crosstalk Impairments in ROADM-Enabled 100G DWDM Networks," Journal of Lightwave Technology, vol. 30, no. 24, pp. 3980–3986, Sept. 2012. DOI: https://doi.org/10.1109/JLT.2012.2208262
M. Seifouri, M. M. Karkhanehchi, and S. Rohani, "Design of Multi-Layer Optical Fibers with Ring Refractive Index to Reduce Dispersion and Increase Bandwidth in Broadband Optical Networks," Engineering, Technology & Applied Science Research, vol. 2, no. 3, pp. 216–220, June 2012. DOI: https://doi.org/10.48084/etasr.166
S. Ilic, B. Jaksic, M. Petrovic, A. Markovic, and V. Elcic, "Analysis of Video Signal Transmission Through DWDM Network Based on a Quality Check Algorithm," Engineering, Technology & Applied Science Research, vol. 3, no. 2, pp. 416–423, Apr. 2013. DOI: https://doi.org/10.48084/etasr.293
S. Zhang et al., "Field and lab experimental demonstration of nonlinear impairment compensation using neural networks," Nature Communications, vol. 10, no. 1, July 2019, Art. no. 3033. DOI: https://doi.org/10.1038/s41467-019-10911-9
A. Singh, N. Gautam, B. Lall, and A. Choudhary, "Restricted Boltzmann Machine Classifier for Nonlinear Compensation in DP-16QAM Single and WDM Coherent Optical Communication Systems," in 2023 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC), June 2023. DOI: https://doi.org/10.1109/CLEO/Europe-EQEC57999.2023.10231617
E. Giacoumidis, A. Tsokanos, M. Ghanbarisabagh, S. Mhatli, and L. P. Barry, "Unsupervised Support Vector Machines for Nonlinear Blind Equalization in CO-OFDM," IEEE Photonics Technology Letters, vol. 30, no. 12, pp. 1091–1094, June 2018. DOI: https://doi.org/10.1109/LPT.2018.2832617
E. Giacoumidis, A. Matin, J. Wei, N. J. Doran, and X. Wang, "Unsupervised Hierarchical Clustering for Blind Nonlinear Equalization in WDM Coherent Optical OFDM," in Asia Communications and Photonics Conference (2017), Nov. 2017, Art. no. Su4C.4. DOI: https://doi.org/10.1364/ACPC.2017.Su4C.4
N. Gautam, S. V. Pendem, B. Lall, and A. Choudhary, "Transformer-Based Nonlinear Equalization for DP-16QAM Coherent Optical Communication Systems," IEEE Communications Letters, vol. 28, no. 3, pp. 577–581, Mar. 2024. DOI: https://doi.org/10.1109/LCOMM.2023.3344996
S. Goossens et al., "On Fiber Nonlinearity Mitigation via 4D Geometric Shaping for Next-Generation Single-Span Systems," IEEE Photonics Technology Letters, vol. 37, no. 6, pp. 349–352, Mar. 2025. DOI: https://doi.org/10.1109/LPT.2025.3544404
Downloads
How to Cite
License
Copyright (c) 2026 M. N. Mahalakshmi, G. Sadashivappa

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
