Hi,
We are observing that when Intel NIC I350 AAP (PCIE Endpoint) is used with Nvidia PCIE Root port, PCIE link fails to enter into L2 states in some cases.
Failure Scenario:
The application software initiates lower power sequence to bring the PCIE link to enter into L2 state by first bringing the endpoint device into D3hot state followed by PME_Turn_Off request from Nvidia Root port to the endpoint device(NIC).
It is observed that the endpoint device(NIC) does bring the link into L1 and also responds with PME_TO_ACK as expected but it does not initiate L2 entry request. So the PCIE Link (Root port) fails to enter into L2 state.
Details of the failure sequence and observations:
There seems to be some issue with the NIC I350 AAP during the L2 entry using Sequence-1 below, whereas same sequence works for the NIC - I350AM4. The Sequence-1 followed for the L2 entry and device details are described below.
Sequence-1 (Failing Sequence): L0 → L1 → L0 → L2/L3 Ready: Intel NIC I350 AAP Failing while NIC - I350AM4 works fine.
1. Device is in L0 and System software directs all Functions of a Downstream component to D3hot.
2. The Downstream component then initiates the transition of the Link to L1 as required.
3. System software then causes the Root Complex to broadcast the PME_Turn_Off Message in preparation for removing the main power source.
4. This Message causes the subject Link to transition back to L0 in order to send it and to enable the Downstream component to respond with PME_TO_Ack.
5. After sending the PME_TO_Ack, the Downstream component initiates the L2/L3 Ready transition protocol. (Intel NIC I350 AAP fails in this step)
Failing Device Details :
I210T1BLK - Single Port NIC ( I350 AAP)
Date of Manufacture : 05/2014
Device 8086:1533 (rev 03)
Working Device Details :
Dual Port NIC - I350AM4
Date of Manufacture : 06/2013
Device 8086:1521 (rev 01)
It is also observed that bothIntel NIC I350 AAP and NIC I350AM4 enter into L2 without any issue while following sequence-2 below.
Sequence-2 (passing sequence):
1. Device is in L0 and system software causes the Root Complex to broadcast the PME_Turn_Off Message in preparation for removing the main power source.
2. The Downstream components respond with PME_TO_Ack.
3. After sending the PME_TO_Ack, the Downstream component initiates the L2/L3 Ready transition protocol.
I am looking for help from your team if there is any known issue with Intel NIC I350 AAP related to Link entering into L2 while following Sequence-1 describe above.
Please also suggest how can we proceed further to root cause the issue and avoid the failure for the L2 entry while following the sequence-1.
Thanks & Regards,
Alap
Baltazar