<?xml version="1.0" encoding="US-ASCII"?>

<?xml-model href="rfc7991bis.rnc"?>

<!-- <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?> -->
<!-- This third-party XSLT can be enabled for direct transformations in XML processors, including most browsers --> version='1.0' encoding='UTF-8'?>

<!DOCTYPE rfc [
  <!ENTITY nbsp    "&#160;">
  <!ENTITY zwsp   "&#8203;">
  <!ENTITY nbhy   "&#8209;">
  <!ENTITY wj     "&#8288;">
]>
<!-- If further character entities are required then they should be added to the DOCTYPE above.
     Use of an external entity file is not recommended. -->

<?rfc strict="yes" ?>
<!-- give errors regarding ID-nits and DTD validation -->

<?rfc compact="yes" ?>
<!-- do not start each main section on a new page -->
<?rfc subcompact="no" ?>
<!-- keep one blank line between list items -->

<rfc category="std" xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-bess-evpn-virtual-eth-segment-19" number="9784" obsoletes="" updates="" consensus="true" submissionType="IETF" ipr="trust200902" tocInclude="true" tocDepth="4" symRefs="true"
    sortRefs="true">

 <!-- ***** FRONT MATTER ***** --> sortRefs="true" version="3" xml:lang="en">

 <front>
   <!-- The abbreviated title is used in the page header - it is only necessary if the
        full title is longer than 39 characters -->
    <title abbrev="EVPN Virtual Ethernet Segment"> EVPN Virtual Segments">Virtual Ethernet Segment Segments for EVPN and Provider Backbone Bridge EVPN </title>
    <seriesInfo name="RFC" value="9784"/>
    <author initials="A" surname="Sajassi" fullname="Ali Sajassi">
      <organization>Cisco Systems</organization>
      <address>
        <email>sajassi@cisco.com</email>
      </address>
    </author>
    <author initials="P" surname="Brissette" fullname="Patrice Brissette">
      <organization>Cisco Systems</organization>
      <address>
        <email>pbrisset@cisco.com</email>
      </address>
    </author>
    <author initials="R" surname="Schell" fullname="Rick Schell">
      <organization>Verizon</organization>
      <address>
        <email>richard.schell@verizon.com</email>
      </address>
    </author>
    <author initials="J" surname="Drake" fullname="John E Drake">
      <organization>Juniper</organization>
      <address>
        <email>jdrake@juniper.net</email>
      </address>
    </author>
    <author initials="J" surname="Rabadan" fullname="Jorge Rabadan">
      <organization>Nokia</organization>
      <address>
        <email>jorge.rabadan@nokia.com</email>
      </address>
    </author>
    <date year="2024"/>
    <area>Routing</area>
    <workgroup>BESS WorkGroup</workgroup> year="2025" month="May"/>
    <area>RTG</area>
    <workgroup>bess</workgroup>

<!-- [rfced] Please insert any keywords (beyond those that appear in
the title) for use on https://www.rfc-editor.org/search.
-->

    <abstract>
      <t>
   Ethernet VPN (EVPN) and Provider Backbone Bridge EVPN (PBB-EVPN) introduce a
   comprehensive suite of solutions for delivering Ethernet services over
   MPLS/IP networks. These solutions offer advanced
   multi-homing
   multihoming capabilities. Specifically, they support Single-Active and
   All-Active redundancy modes for an Ethernet Segment (ES), which is defined
   as a collection of physical links connecting a multi-homed multihomed device or network
   to a set of Provider Edge (PE) devices. This document extends the concept of
   an Ethernet Segment ES by allowing an ES to be associated with a set of Ethernet
   Virtual Circuits (EVCs, (EVCs), such as VLANs) VLANs, or other entities, including MPLS Label
   Switched Paths (LSPs) or Pseudowires pseudowires (PWs). This extended concept is referred
   to as virtual Ethernet Segments (vES). (vESes). This draft list document lists the requirements
   and specifies the necessary extensions to support vES in both EVPN and PBB-EVPN.
      </t>
    </abstract>
  </front>

  <!-- ***** MIDDLE MATTER ***** -->

  <middle>
      <section title="Introduction">
    <section>
      <name>Introduction</name>
      <t> Ethernet VPN (EVPN, (EVPN) <xref target="RFC7432"/>) target="RFC7432"/> and Provider Backbone Bridge EVPN
   (PBB-EVPN,
   (PBB-EVPN) <xref target="RFC7623"/>)) target="RFC7623"/> introduce
   a comprehensive suite of solutions for delivering Ethernet services
   over MPLS/IP networks. These solutions offer advanced
   multi-homing
   multihoming capabilities. Specifically, they support Single-Active and
   All-Active redundancy modes for an Ethernet Segment (ES). As defined in
   <xref target="RFC7432"/>, an Ethernet Segment (ES) ES represents a collection of
   Ethernet links that connect a customer site to one or more PEs Provider Edge (PE) devices. </t>
      <t> This document extends the concept of an Ethernet Segment ES by allowing an ES to be
   associated with a set of Ethernet Virtual Circuits (EVCs, such (EVCs) (such as VLANs) or other
   entities, including MPLS Label Switched Paths (LSPs) or Pseudowires pseudowires (PWs). This
   extended concept is referred to as virtual Ethernet Segments (vES). (vESes). This draft document
   lists the requirements and specifies the necessary extensions to support vES in both EVPN
   and PBB-EVPN. The scope of this document includes PBB-EVPN <xref target="RFC7623"/>,
   EVPN over MPLS <xref target="RFC7432"/>, and EVPN over IP <xref target="RFC8365"/>.
   However, target="RFC8365"/>;
   however, it excludes EVPN over SRv6 <xref target="RFC9252"/>. </t>

  <section title="Requirements Language">
   <t>The
      <section>
        <name>Requirements Language</name>
        <t>
    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
   NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
   "MAY", "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
    NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
    "<bcp14>MAY</bcp14>", and "OPTIONAL" "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as
    described in BCP 14 BCP&nbsp;14 <xref target="RFC2119"/> <xref target="RFC8174"/>
    when, and only when, they appear in all capitals, as shown here.
        </t>
      </section>

  <section title="vESes
      <section>
        <name>vESes in Access Ethernet Networks"> Networks</name>
        <t> Some Service Providers service providers (SPs) seek to extend the concept of physical
   Ethernet links in an ES to encompass Ethernet Virtual Circuits (EVCs), EVCs,
   wherein multiple EVCs (such as VLANs) can be aggregated onto a single
   physical External Network-to-Network Network-Network Interface (ENNI). An ES composed
   of a set of EVCs rather than physical links is referred to as a vES.
   Figure 1
   <xref target="fig-1"/> illustrates two PE devices (PE1 and PE2), each with
   an ENNI aggregating several EVCs. Some of these EVCs on a given ENNI
   can be associated with vESes. For instance, the multi-homed multihomed vES depicted
   in Figure 1 <xref target="fig-1"/> consists of EVC4 on ENNI1 and EVC5 on ENNI2. </t>
          <figure>
            <preamble/>
              <artwork ><![CDATA[
                     3rd Party

<figure anchor="fig-1">
  <name>A Dual-Homed Device/Network (Both SA/AA) That is Single-Homed on the Same ENNI</name>
  <artwork><![CDATA[
                  Third-Party
    +-----+       EAP
    | CE11|EVC1  +---------+
    +-----+   \  |         |       +---+
    Cust. A    \-0=========0--ENNI1|   |
    +-----+      |         |  ENNI1|   |   +-------+   +---+
    | CE12|EVC2--0=========0--ENNI1|PE1|---|       |   |   |
    +-----+      |         |  ENNI1|   |   |SP     |---|PE3|-
                 |       ==0--ENNI1|   |   |IP/MPLS|   |   | \  +---+
    +-----+      |      /  |       +---+   |Core   |   +---+  \-|   |
    | CE22|EVC3--0==== /   |               |Network|            |CE4|
    +-----+      |    X    |               |       |   +---+    |   |
    Cust. B      |   / \   |       +---+   |       |   |   |  /-|   |
    +-----+     -0===   ===0--ENNI2|   |   |       |---|PE4|-/  +---+
    | CE3 |EVC4/ |         |  ENNI2|PE2|---|       |   |   |
    |     |EVC5--0=========0--ENNI2|   |   +-------+   +---+
    +-----+      |         |       +---+
    Cust. C      +---------+   /\
           /\                  ||
           ||                  ENNI
           EVCs             Interface
    <--------802.1Q---------->  <---- EVPN Network -----> <-802.1Q->

   Figure 1: Dual-homed Device/Network (both SA/AA) and SH on same ENNI

                  ]]></artwork>
              <postamble></postamble> <-802.1Q->]]></artwork>
</figure>

        <t> ENNIs are commonly used to reach remote
   customer sites via independent Ethernet access networks or third-
   party third-party
   Ethernet Access Providers (EAP). (EAPs). ENNIs can
   aggregate traffic from many vESes (e.g., hundreds to thousands),
   where each vES is represented by its associated EVC on that ENNI. As a result,
   ENNIs and their associated EVCs are a key element elements of SP external boundaries
   that are carefully designed and closely monitored. As a reminder,
   the ENNI is the demarcation between the SP (IP/MPLS Core Network) core network) and the
   third-party Ethernet Access Provider.
        </t>
        <t> To meet customers' Service Level Agreements (SLA), (SLAs), SPs build
   redundancy via multiple EVPN PEs and across multiple ENNIs (as shown
   in Figure 1) <xref target="fig-1"/>), where a given vES can be multi&nbhy;homed multihomed to two or
   more EVPN
   PE devices (on two or more ENNIs) via their associated EVCs. Just
   like physical ESs in the solutions described in <xref target="RFC7432"/> and <xref target="RFC7623"/> solutions, target="RFC7623"/>, these vESes
   can be single&nbhy;homed single-homed or multi&nbhy;homed ESs multihomed ESs, and when multi&nbhy;homed, then multihomed, they
   can operate in either Single-Active or All-Active redundancy modes.
   In a typical SP external-boundary scenario (e.g., with an EAP), an ENNI can be associated with
   several thousands of single&nbhy;homed single-homed vESes, several hundreds of Single-
   Active vESes Single-Active
   vESes, and it may also be associated with tens or hundreds of All-Active vESes. The specific figures (hundreds, thousands, etc.) used throughout
   this document reflect the relative quantities (hundreds, thousands, etc.) of various elements as understood at the time of writing. </t>
      </section>

             <section title="vESes
      <section>
        <name>vESes in Access MPLS Networks"> Networks</name>
        <t> Other Service Providers (SPs) SPs want to extend the concept of the
   physical links in an ES to individual Pseudowires (PWs) PWs or to MPLS
   Label Switched Paths (LSPs)
   LSPs in Access MPLS networks - networks, i.e., a vES
   consisting of a set of PWs or a set of LSPs. Figure 2 <xref target="fig-2"/> illustrates this concept. </t>
        <figure>
            <preamble/>
                <artwork ><![CDATA[

<figure anchor="fig-2">
  <name>A Dual-Homed and Single-Homed Network on MPLS Aggregation Networks</name>
        <artwork><![CDATA[
                 MPLS Aggregation
                 Network
   +-----+      +-----------------+
   | CE11|EVC1  |                 |
   +-----+   \ +AG1-+  PW1      +-+---+
   Cust. A    -0----|===========|     |
   +-----+     | ---+===========|     |   +-------+   +---+
   | CE12|EVC2-0/   |  PW2   /\ | PE1 +---+       |   |   |
   +-----+     ++---+      /=||=|     |   |       +---+PE3+-
                |         //=||=|     |   |IP/MPLS|   |   | \  +---+
                |        //  \/ +-+---+   |Core   |   +---+  \-+   |
   +-----+EVC3  |    PW3//  LSP1  |       |Network|            |CE4|
   | CE13|    \+AG2-+==//         |       |       |   +---+    |   |
   +-----+     0    |==/PW4  /\ +-+---+   |       |   |   |  /-+   |
               0    |==PW5===||=|     |   |       +---+PE4+-/  +---+
   +-----+    /++---+==PW6===||=| PE2 +---+       |   |   |
   | CE14|EVC4  |            \/ |     |   +-------+   +---+
   +-----+      |           LSP2+-+---+
   Cust. C      +-----------------+
          /\
          ||
          EVCs
   <--802.1Q--> <-----MPLS Agg----> <--- EVPN Network ---> <-802.1Q->

            Figure 2: Dual-Homed and Single-homed Network
                      on MPLS Aggregation networks

                  ]]></artwork>
              <postamble></postamble> <-802.1Q->]]></artwork>
</figure>

        <t> In certain scenarios, Service Providers SPs utilize MPLS Aggregation Networks
   that are managed by separate administrative entities or third-party organizations
   to gain access to their own IP/MPLS core network infrastructure. This situation
   is depicted in Figure 2. <xref target="fig-2"/>. </t>
        <t> In such scenarios, a vES is defined as a set of individual PWs
	when aggregation is not feasible. If aggregation is possible, the vES can be
	associated with a group of PWs that share the same unidirectional LSP pair,
	where the LSP pair consists of the ingress and egress LSPs between the same
	endpoints. </t>
        <t> In the example of Figure 2, <xref target="fig-2"/>, EVC3 is connected to
   a VPWS instance in AG2 that is connected to PE1 and PE2 via PW3 and
   PW5
   PW5, respectively.  EVC4 is connected to another VPWS instance on
   AG2 that is connected to PE1 and PE2 via PW4 and PW6,
   respectively.  Since the PWs for the two VPWS instances can be
   aggregated into the same LSP pair going to and coming from the MPLS
   network, a common vES can be defined for the four mentioned
   PWs. In Figure 2, <xref target="fig-2"/>, LSP1 and LSP2 represent the two LSP pairs between PE1
   and AG2, AG2 and between PE2 and AG2, respectively. The vES consists of these two LSP
   pairs (LSP1 and LSP2) LSP2), and each LSP pair has two PWs. This vES will be
   shared by two separate EVPN instances (e.g., EVI-1 and EVI-2) in
   the EVPN network. PW3 and PW4 are associated with EVI-1 and EVI-2 respectively EVI-2, respectively,
   on PE1, and PW5 and PW6 are associated with EVI-1 and EVI-2 respectively EVI-2, respectively, on PE2. </t>
        <t> In some cases, the aggregation of PWs that share the same LSP pair
   may not be possible. For instance, if PW3 were terminated into a third PE, e.g. e.g.,
   PE3, instead of PE1, the vES would need to be defined on a per
   individual each PW on each PE.</t>
        <t> For MPLS/IP access networks where a vES represents a set of LSP
   pairs or a set of PWs, this document extends the Single-Active multi-homing multihoming
   procedures defined in <xref target="RFC7432"/> and <xref target="RFC7623"/> to
   accommodate vES. The extension of vES to support All-Active multi-homing multihoming in
   MPLS/IP access networks is beyond the scope of this document. </t>
        <t> This draft document defines the concept of a vES and specifies the additional extensions
   necessary to support a vES in accordance with <xref target="RFC7432"/> and
   <xref target="RFC7623"/>. <xref target="requirements"/> enumerates the set of
   requirements for a vES. <xref target="overview"/> details the extensions for a
   vES applicable to EVPN solutions, including those specified in <xref target="RFC7432"/>
   and <xref target="RFC7209"/>. These extensions are designed to meet the requirements
   listed in <xref target="requirements"/>. <xref target="overview"/> also provides
   an overview of the solution, while <xref target="failure"/> addresses failure handling,
   recovery, scalability, and fast convergence of <xref target="RFC7432"/> and
   <xref target="RFC7623"/> for vESes. </t>
      </section>
    </section>

    <section title="Terminology">

    <t>
    <list style="hanging" hangIndent="10">
        <t hangText="AC:">Attachment Circuit</t>
        <t hangText="B-MAC:">Backbone
    <section>
      <name>Terminology</name>
      <dl newline="false" spacing="normal" indent="10">
        <dt>AC:</dt>
        <dd>Attachment Circuit</dd>
        <dt>B-MAC:</dt>
        <dd>Backbone MAC Address </t>
        <t hangText="CE:">Customer Edge Device </t>
        <t hangText="C-MAC:">Customer/Client </dd>
        <dt>CE:</dt>
        <dd>Customer Edge</dd>
        <dt>C-MAC:</dt>
        <dd>Customer/Client MAC Address </t>
        <t hangText="DF:">Designated Forwarder</t>
        <t hangText="ENNI:">External </dd>
        <dt>DF:</dt>
        <dd>Designated Forwarder</dd>
        <dt>ENNI:</dt>
        <dd>External Network-Network Interface </t>
        <t hangText="ES:">Ethernet </dd>
        <dt>ES:</dt>
        <dd>Ethernet Segment </t>
        <t hangText="ESI:">Ethernet </dd>
        <dt>ESI:</dt>
        <dd>Ethernet Segment Identifier </t>
        <t hangText="Ethernet A-D:">Ethernet </dd>
        <dt>Ethernet A-D:</dt>
        <dd>Ethernet Auto-Discovery Route </t>
        <t hangText="EVC:">Ethernet </dd>
        <dt>EVC:</dt>
        <dd>Ethernet Virtual Circuit, Circuit <xref target="MEF63"/> </t>
        <t hangText="EVI:">EVPN </dd>
        <dt>EVI:</dt>
        <dd>EVPN Instance </t>
        <t hangText="EVPN:">Ethernet </dd>
        <dt>EVPN:</dt>
        <dd>Ethernet VPN </t>
        <t hangText="I-SID:">Service </dd>
        <dt>I-SID:</dt>
        <dd>Service Instance Identifier (24 bits and global within a PBB
   network
   network; see <xref target="RFC7080"/>) </t>
        <t hangText="PBB:">Provider target="RFC7080"/>). </dd>
        <dt>MAC:</dt>
        <dd>Media Access Control</dd>
        <dt>PBB:</dt>
        <dd>Provider Backbone Bridge </t>
        <t hangText="PBB-EVPN:">Provider </dd>
        <dt>PBB-EVPN:</dt>
        <dd>Provider Backbone Bridge EVPN </t>
        <t hangText="PE:">Provider Edge Device </t>
        <t hangText="VPWS:">Virtual Pseudowire Service</t>

        <t hangText="Single-Active </dd>
        <dt>PE:</dt>
        <dd>Provider Edge</dd>
        <dt>VPWS:</dt>
        <dd>Virtual Private Wire Service</dd>
        <dt>Single-Active (SA) Redundancy Mode (SA):">When Mode:</dt>
        <dd>When only a single PE, among a
   group of PEs attached to an Ethernet Segment, ES, is allowed to forward
   traffic to/from that Ethernet Segment, then ES, the Ethernet Segment ES is
   defined to be as operating in Single-Active redundancy mode. </t>

        <t hangText="All-Active </dd>
        <dt>All-Active (AA) Redundancy Mode (AA):">When Mode:</dt>
        <dd>When all PEs attached to an Ethernet
   segment, ES are allowed to forward traffic to/from that Ethernet Segment,
   then ES,
   the Ethernet Segment ES is defined to be as operating in All-Active
   redundancy mode. </t>
        </list>
        </t> </dd>
      </dl>
    </section>
    <section title="Requirements" anchor="requirements">
      <name>Requirements</name>
      <t> This section describes the requirements specific to virtual Ethernet
   Segment (vES) vES for (PBB-)EVPN EVPN and PBB-EVPN solutions. These requirements are in
   addition to the ones described in <xref target="RFC8214"/>, <xref target="RFC7432"/>, and
   <xref target="RFC7623"/>. </t>

            <section title="Single-Homed
      <section>
        <name>Single-Homed and Multi-Homed vES"> Multihomed vES</name>
        <t> A PE device MUST <bcp14>MUST</bcp14> support the following types of virtual Ethernet Segments (vES): vESes: </t>

   <t> (R1a) The

	<ol type="(R1%c)">
        <li>The PE MUST <bcp14>MUST</bcp14> handle single-homed vESes on a
        single physical port, such as a single ENNI. </t>

   <t> (R1b) The </li>
        <li>The PE MUST <bcp14>MUST</bcp14> support a combination of
        single-homed vESes and Single-Active
   multi-homed multihomed vESes simultaneously
        on a single physical port, such as a single ENNI.  Throughout this
        document, Single-Active multi-homed multihomed vESes will be referred to as
   Single-Active vESes. </t>

   <t> (R1c) The
        "Single-Active vESes". </li>
        <li>The PE MAY <bcp14>MAY</bcp14> support All-Active multi-homed multihomed
        vESes on a single physical port.  Throughout this document, All-Active multi-homed
        multihomed vESes will be referred to as
   All-Active vESes. </t>

   <t> (R1d) The "All-Active vESes". </li>
        <li>The PE MAY <bcp14>MAY</bcp14> support a combination of
        All-Active vESes along with other types of vESes on a single physical
        port. </t>

   <t> (R1e) A Multi-Homed </li>
        <li>A multihomed vES, whether Single-Active or All-Active, can
        span across two or more ENNIs on any two or more PEs. </t> </li>
	</ol>
      </section>

            <section title="Local Switching">
      <section>
        <name>Local Switching</name>
        <t> Many vESes of different types can be aggregated on a single physical
   port on a PE device and some of these vESes can belong to the same
   service instance (e.g., EVI). This translates into the need for
   supporting local switching among the vESes for the same service
   instance on the same physical port (e.g., ENNI) of the PE. </t>

   <t> (R3a) A

<ol type="(R2%c)">
        <li>A PE device that supports the vES function MUST <bcp14>MUST</bcp14> support local switching among different vESes associated with the same service instance on a single physical port. For instance, in Figure 1, <xref target="fig-1"/>, PE1 must support local switching between CE11 and CE12, which are mapped to two single-homed vESes on ENNI1. In the case of Single-Active vESes, the local switching is performed among active EVCs associated with the same service
   instance on the same ENNI. </t> </li>
</ol>
      </section>

            <section title="EVC
      <section>
        <name>EVC Service Types"> Types</name>
        <t> A physical port, such as an ENNI of a PE device, can aggregate numerous
   EVCs, each associated with a vES. An EVC may carry one or more VLANs. Typically,
   an EVC carries a single VLAN and is therefore associated with a single broadcast
   domain. However, there are no restrictions preventing an EVC from carrying
   multiple VLANs. </t>

   <t> (R4a) An

   <ol type="(R3%c)" group="req4">
        <li>An EVC can be associated with a single broadcast domain, such as in
   a VLAN-based service or a VLAN bundle service. </t>

   <t> (R4b) An </li>
        <li>An EVC MAY <bcp14>MAY</bcp14> be associated with several broadcast domains, such as in
   a VLAN-aware bundle service. </t> </li>
   </ol>
        <t> Similarly, a PE can aggregate multiple LSPs and PWs. In the case of individual
   PWs per vES, typically, a PW is typically associated with a single broadcast domain, although
   there are no restrictions preventing a PW from carrying multiple VLANs if the PW
   is configured in Raw mode. </t>

   <t> (R4c) A
   <ol type="(R3%c)" group="req4">
        <li>A PW can be associated with a single broadcast domain, such as in a
   VLAN-based service or a VLAN bundle service. </t>

   <t> (R4d) An </li>
        <li>A PW MAY <bcp14>MAY</bcp14> be associated with several broadcast domains, such as in a
   VLAN-aware bundle service. </t> </li>
   </ol>
      </section>

            <section title="Designated
      <section>
        <name>Designated Forwarder (DF) Election">

   <t> Section 8.5 of <xref target="RFC7432"/> Election</name>
        <t><xref section="8.5" target="RFC7432" sectionFormat="of"/> specifies the default procedure for DF
   election in EVPN, which is also applied in <xref target="RFC7623"/> and
   <xref target="RFC8214"/>. <xref target="RFC8584"/> elaborates on additional
   procedures for DF election in EVPN. These DF election procedures are performed at
   the granularity of (ESI, Ethernet Tag). In the context of a vES, the same EVPN default
   procedure for DF election is applicable, applicable but at the granularity of (vESI, Ethernet Tag).
   In this context, the Ethernet Tag is represented by an I-SID in PBB-EVPN and by a
   VLAN ID (VID) in EVPN.

   As described in <xref target="RFC7432"/>, this default procedure for DF election at
   the granularity of (vESI, Ethernet Tag) is also known as "service carving." The goal
   of service carving is to evenly distribute the DFs for different vESes among various PEs,
   thereby ensuring an even distribution of traffic across the PEs. The following
   requirements are applicable to the DF election of vESes for (PBB-)EVPN. EVPN and PBB-EVPN. </t>

   <t> (R5a) A
<ol type="(R4%c)">
        <li>A PE that supports vES function, MUST function <bcp14>MUST</bcp14> support a vES with m EVCs among
   n ENNIs belonging to p PEs in any arbitrary order; order, where n >= &gt;= p >= &gt;= m >=2. &gt;=2.
   For example, if there is a vES with 2 EVCs and there are 5 ENNIs on 5 PEs
   (PE1 through PE5), then vES can be dual homed dual-homed to PE2 and PE4 PE4, and the DF
   election must be performed between PE2 and PE4.</t>

   <t> (Rbc) Each PE4.</li>
        <li>Each vES MUST <bcp14>MUST</bcp14> be identified by its own virtual ESI (vESI). </t> </li>
</ol>
      </section>

            <section title="EVC Monitoring">
      <section>
        <name>EVC Monitoring</name>
        <t> To detect the failure of an individual EVC and subsequently perform DF
   election for its associated vES as a result of this failure, each EVC should
   be monitored independently. </t>

   <t> (R6a) Each

<ol type="(R5%c)">
        <li>Each EVC SHOULD <bcp14>SHOULD</bcp14> be independently monitored for its operational health. </t>

   <t> (R6b) A </li>
        <li>A failure in a single EVC, among many aggregated on a single physical
   port or ENNI, MUST <bcp14>MUST</bcp14> trigger a DF election for its associated vES. </t> </li>
</ol>
      </section>

                <section title="Failure
      <section>
        <name>Failure and Recovery">

   <t> (R7a) Failure Recovery</name>
<ol type="(R6%c)">
        <li>Failure and failure recovery of an EVC for a Single-homed single-homed vES
   SHALL NOT
   <bcp14>SHALL NOT</bcp14> impact any other EVCs within its service instance or any
   other service instances. In other words, for PBB-EVPN, it SHALL NOT <bcp14>SHALL NOT</bcp14>
   trigger any MAC flushing both within both its own I-SID as well as and other
   I-SIDs. </t>

   <t> (R7b) In </li>
        <li>In case of All-Active vES, failure and failure
   recovery of an EVC for that vES SHALL NOT <bcp14>SHALL NOT</bcp14> impact any other EVCs within
   its service instance or any other service instances. In other
   words, for PBB-EVPN, it SHALL NOT <bcp14>SHALL NOT</bcp14> trigger any MAC flushing both
   within both its own I-SID as well as and other I-SIDs. </t>

   <t> (R7c) Failure </li>
        <li>Failure and failure recovery of an EVC for a Single-Active vES
   SHALL
   <bcp14>SHALL</bcp14> impact only its own service instance. In other words, for PBB-
   EVPN, PBB-EVPN,
   MAC flushing SHALL <bcp14>SHALL</bcp14> be limited to the associated I-SID only and
   SHALL NOT
   <bcp14>SHALL NOT</bcp14> impact any other I-SIDs. </t>

   <t> (R7d) Failure </li>
        <li>Failure and failure recovery of an EVC for a Single-Active vES
   MUST
   <bcp14>MUST</bcp14> only impact C-MACs associated with multi-homed a multihomed device/network for that service
   instance. In other words, MAC flushing MUST <bcp14>MUST</bcp14> be limited to a single
   service instance (I-SID in the case of PBB-EVPN) and only C-MACs for a
   Single-Active multi-homed multihomed device/network. </t> </li>
</ol>
      </section>

                <section title="Fast Convergence">
      <section>
        <name>Fast Convergence</name>

        <t> Since many EVCs (and their associated vESes) are
   aggregated via a single physical port (e.g., ENNI), then the when there is a failure
   of that physical port port, it impacts many vESes and triggers equally triggers
   many ES route withdrawals. Formulating, sending,
   receiving, and processing such large number numbers of BGP messages can
   introduce delay in DF election and convergence time. As such, it is
   highly desirable to have a mass&nbhy;withdraw mass-withdraw mechanism similar to the one
   in <xref target="RFC7432"/> for withdrawing many Ethernet A-D per ES routes. </t>

   <t> (R8a) There SHOULD
   <ol type="(R7%c)">
     <li>There <bcp14>SHOULD</bcp14> be a mechanism equivalent to EVPN mass&nbhy;withdraw mass withdraw
	such that upon an ENNI failure, only a single BGP message is needed
   to indicate to the remote PEs is needed to trigger DF election for all impacted
   vES vESes associated with that ENNI. </t> </li>
   </ol>
      </section>
    </section>
    <section title="Solution Overview" anchor="overview">
      <name>Solution Overview</name>
      <t> The solutions described in <xref target="RFC7432"/> and <xref target="RFC7623"/>
      are leveraged
   as&nbhy;is
   as is, with the modification that the ESI assignment is
   performed for an EVC or a group of EVCs or LSPs/PWs instead of a link or a group of
   physical links. In other words, the ESI is associated with a virtual
   ES (vES), hereby vES (hereby referred to as vESI. the "vESI"). </t>
      <t> In the EVPN solution, the overall procedures remain consistent,
   with the primary difference being the handling of physical port failures
   that can affect multiple vESes. Sections <xref target="fail_evc_sa_evpn" format="counter"/>
   and <xref target="fail_port_sa_evpn" format="counter"/> describe the
   procedures for managing physical port or link failures in the context of EVPN.
   In a typical multi-homed multihomed setup, MAC addresses learned behind a vES are
   advertised using the ESI associated with the vES, referred to as vES (i.e., the vESI. vESI).
   EVPN aliasing and mass-withdraw operations are conducted with respect to the
   vES identifier. Specifically, the Ethernet Auto-Discovery (A-D) A-D routes for these
      operations are advertised using the vESI instead of the ESI. </t>

      <t> For the PBB-EVPN solution, the main change is with respect to the B-MAC
   address assignment assignment, which is performed in a similar way to what is described in
   section 7.2.1.1 of
   <xref target="RFC7623"/> section="6.2.1.1" target="RFC7623" sectionFormat="of"/>, with the following refinements:

            <list style="symbols">

      </t>
      <ul spacing="normal">
        <li>
          <t> One shared B-MAC address SHOULD <bcp14>SHOULD</bcp14> be used per PE for the single&nbhy;homed single-homed
   vESes. In other words, a single B-MAC is shared for all single&nbhy;homed single-homed
   vESes on that PE. </t>
        </li>
        <li>
          <t> One shared B-MAC address SHOULD <bcp14>SHOULD</bcp14> be used per PE PE, per physical port
   (e.g., ENNI) for the Single-Active vESes. In other words, a single
   B-MAC is shared for all Single-Active vESes that share the same ENNI. </t>
        </li>
        <li>
          <t> One shared B-MAC address MAY <bcp14>MAY</bcp14> be used for all Single-Active vESes on
   that PE. </t>
        </li>
        <li>
          <t> One B-MAC address SHOULD <bcp14>SHOULD</bcp14> be used per set of EVCs representing an
   All-Active vES.  In other words, a single B-MAC address is
   used per vES for All-Active scenarios. </t>
        </li>
        <li>
          <t> A single B-MAC address MAY <bcp14>MAY</bcp14> also be used per vES vES, per PE for Single-
   Active Single-Active scenarios. </t>

              </list>
          </t>
        </li>
      </ul>
      <section title="EVPN anchor="df_election">
        <name>EVPN DF Election for vES" anchor="df_election"> vES</name>
        <t>
   The procedure for service carving procedures for vESes is are
   almost the same as the ones procedures outlined in section 8.5 of <xref target="RFC7432"/> section="8.5" target="RFC7432" sectionFormat="of"/>
   and in <xref target="RFC8584"/> target="RFC8584"/>, except for the fact that ES is replaced with vES.</t>
        <t>For the sake of clarity and completeness, the default DF election
   procedure of <xref target="RFC7432"/> is repeated below with the necessary
   changes:

                    <list style="numbers">

        </t>
        <ol spacing="normal" type="1">
	  <li>
            <t> When a PE discovers the vESI or is configured with the vESI
            associated with its attached vES, it advertises an Ethernet Segment ES route with the associated ES-Import extended community
            attribute. </t>
          </li>
          <li>
            <t> The PE then starts a timer (default value = 3 seconds) to
            allow the reception of Ethernet Segment ES routes from other PE nodes
            connected to the same vES. This timer value MUST <bcp14>MUST</bcp14> be
            the same across all PEs connected to the same vES. </t>
          </li>
          <li>
            <t> When the timer expires, each PE builds an ordered list of the
            IP addresses of all the PE nodes connected to the vES (including
            itself), in increasing numeric value. Each IP address in this list
            is extracted from the "Originator Router's IP address" field of
            the advertised Ethernet Segment ES route. Every PE is then given an
            ordinal indicating its position in the ordered list, starting with
            0 as the ordinal for the PE with the numerically lowest IP
            address. The ordinals are used to determine which PE node will be
            the DF for a given EVPN instance on the vES using the following
            rule: Assuming a redundancy group of N PE nodes, the PE with
            ordinal i is the DF for an EVPN instance with an associated
            Ethernet Tag value of V when (V mod N) = i.  It should be noted
            that using the "Originator Router's IP address" field in the Ethernet Segment ES
            route to get the PE IP address needed for the ordered list,
            list allows for a CE to be multi&nbhy;homed multihomed across different ASes ASes, if
            such need ever arises. </t>
          </li>
          <li>
            <t> The PE that is elected as a DF for a given EVPN instance will
            unblock traffic for that EVPN instance. Note that the DF PE
            unblocks all traffic in both ingress and egress directions for
            Single-Active
   vES vESes and unblocks multi&nbhy;destination multi-destination in the egress
            direction for All-Active
   Multi-homed vES. multihomed vESes. All non-DF PEs block all
            traffic in both ingress and egress directions for Single-Active vES
            vESes and block multi&nbhy;destination multi-destination traffic in the egress direction
            for All-Active vES. </t>
                    </list> vESes. </t>
          </li>
        </ol>
        <t> In case of an EVC failure, the affected PE withdraws its corresponding Ethernet
   Segment ES route if there are no more EVCs associated to the vES in the
   PE. This will re-trigger the DF Election election procedure on all the PEs in
   the Redundancy Group. redundancy group. For PE node failure, or upon PE commissioning
   or decommissioning, the PEs re-trigger the DF Election election procedure
   across all affected vESes.
   In case of a Single-Active, Single-Active scenario,
   when a service moves from one PE in the Redundancy Group redundancy group to another
   PE because of DF re-election, the PE, which PE (which ends up being the
   elected DF for the service, MUST service) <bcp14>MUST</bcp14> trigger a MAC address flush
   notification towards the associated vES if the multi-homing multihoming device is a bridge
   or the multi-homing multihoming network is an Ethernet bridged network.</t>
        <t> For LSP-based and PW-based vES, the non-DF PE SHOULD <bcp14>SHOULD</bcp14> signal PW-status
   'standby' to the Aggregation PE (e.g., AG1 and AG2 in Figure 2), <xref target="fig-2"/>),
   and a new DF PE MAY <bcp14>MAY</bcp14> send an LDP a Label Distribution Protocol (LDP) MAC withdraw message as a MAC
   address flush notification. It should be noted that the PW-status is
   signaled for the scenarios where there is a one-to-one mapping
   between EVI (EVPN instance) and the PW.  </t>
      </section>
      <section title="Grouping anchor="grouping">
        <name>Grouping and Route Coloring for vES" anchor="grouping"> vES</name>
        <t>Physical ports (e.g. (e.g., ENNI) which that aggregate many EVCs
        are 'colored' to enable the grouping schemes described below. </t>
        <t>By default, the MAC address of the corresponding port (e.g. (e.g., ENNI)
        is used to represent the 'color' of the port, and the
        EVPN Router's MAC Extended Community defined
        in <xref target="RFC9135"/> is used to signal this color.</t>
        <t>The difference between coloring mechanism mechanisms for EVPN and PBB-EVPN is that
        for EVPN,
        the extended community is advertised with the Ethernet A-D per ES
        route whereas for PBB-EVPN, EVPN, whereas the extended community is advertised
        with the B-MAC route.</t> route for PBB-EVPN.</t>
        <t> The subsequent sections detailing Grouping of Ethernet Auto-Discovery (A-D) A-D
	    per ES routes and Grouping of B-MAC addresses will be essential for addressing port
	    failure handling, as discussed in Sections <xref target="fail_port_sa_evpn"/>, target="fail_port_sa_evpn" format="counter"/>,
	    <xref target="fail_port_sa_pbbevpn"/>, target="fail_port_sa_pbbevpn" format="counter"/>, and <xref target="convergence"/>. target="convergence" format="counter"/>. </t>
        <section title="EVPN anchor="grouping_evpn">
          <name>EVPN Route Coloring for vES" anchor="grouping_evpn"> vES</name>
          <t>When a PE discovers the vESI or is configured with the vESI associated
            with its attached vES, an Ethernet-Segment ES route and Ethernet A-D per ES
            route are generated using the vESI identifier.</t>
          <t>These Ethernet-Segment ES and Ethernet A-D per ES routes specific to each
            vES are colored with an attribute representing their association
            to a physical port (e.g. (e.g., ENNI).</t>
          <t>The corresponding port 'color' is encoded in the
            EVPN Router's MAC Extended Community defined in <xref target="RFC9135"/>
            and advertised along with the Ethernet Segment ES and Ethernet A-D per ES
            routes for this vES. The color (which is the MAC address of the port) MUST <bcp14>MUST</bcp14>
            be unique. </t>
          <t>The PE also constructs a special Grouping Ethernet A-D per ES route
            which
            that represents all the vES vESes associated with the port (e.g. (e.g., ENNI).
            The corresponding port 'color' is encoded in the ESI field.
            For this encoding, Type 3 ESI (<relref (<xref target="RFC7432" section="5"/>) is used
            with the MAC field set to the color (MAC address) of the port
            and the 3-octet local discriminator field set to 0xFFFFFF.
          </t>
          <t>The ESI label extended community (<relref (<xref target="RFC7432" section="7.5"/>)
            is not relevant to Grouping Ethernet A-D per ES route. The label value is not
            used for encapsulating BUM (Broadcast, Unknown-unicast, Multicast) Broadcast, Unknown Unicast, and Multicast (BUM) packets
            for any split-horizon function. The ESI
            label extended community MUST NOT <bcp14>MUST NOT</bcp14> be added to Grouping Ethernet A-D per ES route and
            MUST
            <bcp14>MUST</bcp14> be ignored on receiving the PE.
          </t>
          <t> The Grouping Ethernet Auto-Discovery (A-D) A-D per ES route is advertised with a
	        list of Route Targets corresponding to the affected service instances. If the
	        number of associated Route Targets exceeds the capacity of a single route,
	        multiple Grouping Ethernet A-D per ES routes are advertised accordingly as
	        specified in Section 8.2 of <xref target="RFC7432"/>. section="8.2" target="RFC7432" sectionFormat="of"/>. </t>
        </section>
        <section title="PBB-EVPN anchor="grouping_pbbevpn">
          <name>PBB-EVPN Route Coloring for vES" anchor="grouping_pbbevpn"> vES</name>
          <t> In PBB-EVPN, particularly when there are a large number numbers of service instances
	        (i.e., I-SIDs) associated with each EVC, the PE device MAY <bcp14>MAY</bcp14> assign a color attribute
	        to each vES B-MAC route, indicating their association with a physical port
	        (e.g., an ENNI). </t>
          <t>The corresponding port 'color' is encoded in the
            EVPN Router's MAC Extended Community defined
            in <xref target="RFC9135"/> and advertised
            along with the B-MAC for this vES in PBB-EVPN.</t>
          <t>The PE MAY then <bcp14>MAY</bcp14> also construct a special Grouping B-MAC route
            which
            that represents all the vES vESes associated with the port (e.g. (e.g., ENNI).
            The corresponding port 'color' is encoded directly into this
            special Grouping B-MAC route.</t>
        </section>
      </section>
    </section>
    <section title="Failure anchor="failure">
      <name>Failure Handling and Recovery" anchor="failure"> Recovery</name>
      <t> There are several failure scenarios to consider such as:
    <list style="hanging" hangIndent="3">
       <t hangText="A:">CE
      </t>
      <dl newline="false" spacing="normal" indent="3">
        <dt>A:</dt>
        <dd>CE uplink port failure</t>
       <t hangText="B:">Ethernet failure</dd>
        <dt>B:</dt>
        <dd>Ethernet Access Network failure</t>
       <t hangText="C:">PE failure</dd>
        <dt>C:</dt>
        <dd>PE access-facing port or link failure</t>
       <t hangText="D:">PE failure</dd>
        <dt>D:</dt>
        <dd>PE node failure</t>
       <t hangText="E:">PE failure</dd>
        <dt>E:</dt>
        <dd>PE isolation from IP/MPLS network</t>
    </list>
    </t> network</dd>
      </dl>
      <t>
   The solutions specified in <xref target="RFC7432"/>, <xref target="RFC7623"/>,
   and <xref target="RFC8214"/> provide protection against failures as described in
   these respective references. In the context of these solutions, the presence of vESes
   introduces an additional failure scenario beyond those already considered, specifically
   the failure of individual EVCs. Addressing vES failure scenarios necessitates the
   independent monitoring of EVCs or PWs. Upon detection of failure or service restoration,
   appropriate DF election and failure recovery mechanisms must be executed. </t>
      <t> <xref target="RFC7023"/> is used for monitoring EVCs EVCs, and upon failure detection of a
   given EVC, the DF election procedure per <xref target="df_election"/> is executed. For
   PBB-EVPN, some extensions are needed to handle the failure and
   recovery procedures of <xref target="RFC7623"/> to meet the above
   requirements. These extensions are described in the next section. </t>
      <t> <xref target="RFC4377"/> and <xref target="RFC6310"/> are used for monitoring the status of LSPs
   and/or PWs associated to vES. </t>

        <figure>
            <preamble/>
                <artwork ><![CDATA[

<figure anchor="fig-3">
  <name>Failure Scenarios A, B, C, D, and E</name>
      <artwork><![CDATA[
                      B            D
                      ||           ||
                      \/           \/
                    +-----+
       +-----+      |     |       +---+
       | CE1 |EVC2--0=====0--ENNI1|   |   +-------+
       +-----+      |    =0--ENNI1|PE1|---|       |  +---+  +---+
       Cust. A      |   / |       |   |   |IP/MPLS|--|PE3|--|CE4|
       +-----+      |  /  |       +---+   |Network|  |   |  +---+
       |     |EVC2--0==   |               |       |  +---+
       | CE2 |      |     |       +---+   |       |
       |     |EVC3--0=====0--ENNI2|PE2|---|       |
       +-----+      |     |       |   |   +-------+
                    +-----+       +---+
              /\                /\     /\
              ||                ||     ||
              A                 C      E

   Figure 4: Failure Scenarios A,B,C,D and E

                ]]></artwork>
            <postamble></postamble>      E]]></artwork>
</figure>
      <section title="EVC anchor="fail_evc_sa_evpn">
        <name>EVC Failure Handling for Single-Active vES in EVPN" anchor="fail_evc_sa_evpn"> EVPN</name>
        <t>
   In <xref target="RFC7432"/>, when a DF PE connected to a Single-Active multi&nbhy;homed Ethernet
   Segment multihomed ES loses connectivity to the segment, due to link or port
   failure, it signals to the remote PEs to invalidate all MAC addresses
   associated with that Ethernet Segment. ES. This is done by means of a
   mass&nbhy;withdraw
   mass-withdraw message, by withdrawing the Ethernet A-D per ES route.
   It should be
   noted that for dual-homing use cases where there is only a single
   backup path, MAC invalidating can be avoided by the remote PEs as they
   can update their next hop associated with the affected MAC
   entries to the backup path per the procedure described in section 8.2 of <xref section="8.2" sectionFormat="of" target="RFC7432"/>.
        </t>
        <t>
   In case of an EVC failure which that impacts a single vES, this same
   EVPN procedure is used. In this case, the mass&nbhy;withdraw mass withdraw is conveyed
   by withdrawing the Ethernet A-D per vES route carrying the vESI representing
   the failed EVC. The remote PEs upon Upon receiving this
   message
   message, the remote PEs perform the same procedures outlined in section 8.2 of <xref section="8.2" sectionFormat="of" target="RFC7432"/>.
        </t>
      </section>
      <section title="EVC anchor="fail_evc_sa_pbbevpn">
        <name>EVC Failure Handling for a Single-Active vES in PBB-EVPN" anchor="fail_evc_sa_pbbevpn"> PBB-EVPN</name>
        <t> In <xref target="RFC7432"/> target="RFC7432"/>, when a PE connected to a Single-Active Ethernet Segment ES
   loses connectivity to the segment, due to link or port failure, it
   signals the remote PE to flush all C-MAC addresses associated with
   that Ethernet Segment. ES. This is done by updating the advertised a B-MAC route's
   MAC Mobility Extended extended community. </t>
        <t> In case of an EVC failure that impacts a single vES, if the above
   PBB-EVPN procedure is used, it results in excessive C-MAC flushing
   because a single physical port can support a large number of EVCs (and
   their associated vESes) and thus vESes); therefore, updating the advertised B-MAC corresponding to
   the physical port, with MAC mobility Extended Mobility extended community, will result in
   flushing C-MAC addresses not just for the impacted EVC but for all
   other EVCs on that port.</t>
        <t> To reduce the scope of C-MAC flushing to only the impacted
   service instances (the service instance(s) impacted by the EVC
   failure), the PBB-EVPN C-MAC flushing needs to be adapted on a per service
   instance per-service-instance
   basis (i.e., per I-SID).

   <xref target="RFC9541"/> introduces a B-MAC/I-SID route where the
   existing PBB-EVPN B-MAC route is modified to carry an I-SID I-SID, instead of a NULL value, in the "Ethernet Tag ID"
   field instead of NULL value. This field indicates to field. To the receiving PE, to flush this field indicates flushing all
   C-MAC addresses associated with that I-SID for that B-MAC. This C-MAC flushing mechanism per I-SID
   SHOULD
   <bcp14>SHOULD</bcp14> be used in case of an EVC failure impacting a vES. Since typically an EVC typically maps to a single
   broadcast domain and thus, thus a single service instance, the affected PE only needs to
   advertise a single B-MAC/I-SID route.
   However, if the failed EVC carries multiple
   VLANs each with its own broadcast domain, then the affected PE needs to advertise multiple
   B-MAC/I-SID routes - routes, i.e., one route for each VLAN (broadcast domain) - i.e., domain), meaning one route for each I-SID.
   Each B-MAC/I-SID B&nbhy;MAC/I-SID route basically instructs the remote PEs to perform flushing for
   C-MACs corresponding to the advertised B-MAC only for the advertised I-SID.</t>
        <t> The C-MAC flushing based on a B-MAC/I-SID route works fine when there are only a few
   VLANs (e.g., I-SIDs) per EVC. However However, if the number of I-SIDs associated with a
   failed EVC is large, then it is RECOMMENDED <bcp14>RECOMMENDED</bcp14> to assign a B-MAC per vES vES, and upon EVC failure,
   the affected PE simply withdraws this B-MAC message to other PEs. </t>
      </section>
      <section title="Port anchor="fail_port_sa_evpn">
        <name>Port Failure Handling for Single-Active vESes in EVPN" anchor="fail_port_sa_evpn"> EVPN</name>
        <t> When many EVCs are aggregated via a single physical port
   on a PE, where each EVC corresponds to a vES, then the port failure
   impacts all the associated EVCs and their corresponding vESes. If the
   number of EVCs corresponding to the Single-Active vESes for that
   physical port is in the thousands, then thousands of service instances
   are impacted. Therefore, the propagation of failure in BGP needs
   to address all these impacted service instances. In order to achieve this,
   the following extensions are added to the baseline EVPN mechanism:

            <list style="numbers">

        </t>
        <ol spacing="normal" type="1"><li>
            <t> The PE MAY <bcp14>MAY</bcp14> color each Ethernet A-D per ES route
            for a given vES, as described in <xref
            target="grouping_evpn"/>. The PE SHOULD <bcp14>SHOULD</bcp14> use the
            MAC physical port MAC by default.  The receiving PEs take note of this
            color and create a list of vESes for this color.</t>
          </li>
          <li>
            <t>The PE MAY advertises <bcp14>MAY</bcp14> advertise a special Grouping
            Ethernet A-D per ES route for that color, which represents all the vES
            vESes associated with the port.</t>
          </li>
          <li>
            <t> Upon a port failure (e.g., an ENNI failure), the PE MAY
            <bcp14>MAY</bcp14> send a mass&nbhy;withdraw mass-withdraw message by withdrawing the
            Grouping Ethernet A-D per ES route.</t>
          </li>
          <li>
          <t> When this message is received, the remote PE MAY
            <bcp14>MAY</bcp14> detect the special vES mass&nbhy;withdraw mass-withdraw message by
            identifying the Grouping Ethernet A-D per ES route. The remote PEs MAY
            <bcp14>MAY</bcp14> then access the list created
   in (1) of the vESes created
            per item 1 for the specified color, color and initiate locally initiate MAC address
            invalidating procedures for each of the vESes in the list. </t>
               </list>
          </li>
        </ol>
        <t>

   In scenarios where a logical ENNI is used used, the above procedure equally
   applies. The logical ENNI is represented by a Grouping Ethernet A-D per ES
   route where the Type 3 ESI and the 6 bytes used in the ENNI's ESI MAC address
   field is are used as a color for the vESes as described above
   and in <xref target="grouping_evpn"/>.</t>
      </section>
      <section title="Port anchor="fail_port_sa_pbbevpn">
        <name>Port Failure Handling for Single-Active vESes in PBB-EVPN" anchor="fail_port_sa_pbbevpn"> PBB-EVPN</name>
        <t>When many EVCs are aggregated via a single physical port
   on a PE, where each EVC corresponds to a vES, then the port failure
   impacts all the associated EVCs and their corresponding vESes. If the
   number of EVCs corresponding to the Single-Active vESes for that
   physical port is in the thousands, then thousands of service instances (I-SIDs) are impacted. In such failure scenarios, the following two
   MAC flushing mechanisms per <xref target="RFC7623"/> can be performed.

            <list style="numbers">

        </t>
        <ol spacing="normal" type="1"><li>
            <t> If the MAC address of the physical port is used for PBB
            encapsulation as B-MAC SA, then upon the port failure, the PE MUST
            <bcp14>MUST</bcp14> use the EVPN MAC route withdrawal message to
            signal the flush.</t>
          </li>
          <li>
            <t> If the PE PE's shared MAC address is used for PBB encapsulation as
            B-MAC SA, then upon the port failure, the PE MUST <bcp14>MUST</bcp14>
            re-advertise this MAC route with the MAC Mobility Extended Community extended
            community to signal the flush.</t>

               </list>
          </li>
        </ol>
        <t>

   The first method is recommended because it reduces the scope of
   flushing the most.
        </t>
        <t>As noted above, the
   advertisement of the extended community along with the B-MAC route for coloring purposes is optional
   and only recommended when there are many vESes per physical port and each vES is associated with
   a very large number of service instances (i.e., a large number of I-SIDs). </t>
        <t> If there are large number numbers of service instances (i.e., I-SIDs)
   associated with each EVC, and if there is a B-MAC assigned per vES
   as recommended in the above section, then in order to handle
   port failure efficiently, the following extensions are added
   to the baseline PBB-EVPN mechanism:

            <list style="numbers">

        </t>
        <ol spacing="normal" type="1">
	  <li>
            <t>Each vES MAY <bcp14>MAY</bcp14> be colored with a MAC address representing the
   physical port like the coloring mechanism for EVPN.
   In other words, each B-MAC representing a vES is advertised
   with the 'color' of the physical port per <xref target="grouping_pbbevpn"/>.

   The receiving PEs take note of this color being advertised
   along with the B-MAC route route, and for each such color, they
   create a list of vESes associated with this color.</t>
          </li>
          <li>
            <t>The PE MAY <bcp14>MAY</bcp14> advertise a special Grouping B-MAC route
   for that color (consisting by default of a port MAC address), address by default),
   which represents all the vES vESes associated with the port.</t>
          </li>
          <li>
            <t> Upon a port failure (e.g., ENNI failure), the PE MAY <bcp14>MAY</bcp14> send a mass&nbhy;withdraw mass-withdraw
   message by withdrawing the Grouping B-MAC route.</t>
          </li>
          <li>
            <t> When this message is received, the remote PE MAY <bcp14>MAY</bcp14>
   detect the special vES mass&nbhy;withdraw mass-withdraw message by identifying the
   Grouping B-MAC route. The remote PEs MAY <bcp14>MAY</bcp14> then access the list created
   in (1) above for the specified color, color and flush all C-MACs associated with the failed physical port. </t>

            </list>
        </t>
          </li>
        </ol>
      </section>
      <section title="Fast anchor="convergence">
        <name>Fast Convergence in (PBB-)EVPN" anchor="convergence"> EVPN and PBB-EVPN</name>
        <t>As described above, when many EVCs are aggregated via a
   physical port on a PE, and where each EVC corresponds to a vES, then the
   port failure impacts all the associated EVCs and their corresponding
   vESes. Two actions must be taken as the result of such a port failure:

            <list style="symbols">

        </t>
        <ul spacing="normal">
          <li>
            <t> For EVPN EVPN, initiate mass&nbhy;withdraw the mass-withdraw procedure for all vESes associated with
   the failed port to invalidate MACs and for PBB-EVPN to flush all C-MACs associated with
   the failed port across all vESes and the impacted I-SIDs </t>
          </li>
          <li>
            <t> Use DF election for all impacted vESes associated with the failed port </t>
            </list>
          </li>
        </ul>
        <t>

   <xref target="fail_port_sa_evpn"/> already describes how to perform mass&nbhy;withdraw a mass withdraw
   for all affected vESes and invalidating invalidate MACs using a single BGP withdrawal
   of the Grouping Ethernet A-D per ES route.
   <xref target="fail_port_sa_pbbevpn"/> describes how to only flush C-MAC address addresses
   associated with the failed physical port (e.g., optimum C-MAC flushing)
   as well as, optionally, the withdrawal of a Grouping B-MAC route.</t>
        <t>This section describes how to perform DF election in the most
   optimal way - way, e.g., to trigger by triggering DF election for all impacted vESes
   (which can be very large) among the participating PEs via a single
   BGP message as opposed to sending a large number of BGP messages (one per
   vES). This section assumes that the MAC flushing mechanism described in
   <xref target="fail_port_sa_pbbevpn"/> is used and route coloring is are used.
        </t>

		<figure>
			<preamble/>
				<artwork ><![CDATA[
<figure anchor="fig-4">
  <name>Fast Convergence Upon ENNI Failure</name>
        <artwork><![CDATA[
                  +-----+
       +----+     |     |       +---+
       | CE1|AC1--0=====0--ENNI1|   |  +-------+
       |    |AC2--0     |       |PE1|--|       |
       +----+     |\  ==0--ENNI2|   |  |       |
                  | \/  |       +---+  |       |
                  | /\  |              |IP/MPLS|
       +----+     |/  \ |       +---+  |Network|   +---+  +---+
       | CE2|AC4--0    =0--ENNI3|   |  |       |---|PE4|--|CE4|
       |    |AC4--0=====0--ENNI3|PE2|--|       |   +---+  +---+
       +----+     | ====0--ENNI3|   |  |       |
                  |/    |       +---+  |       |
                  0     |              |       |
       +----+    /|     |       +---+  |       |
       | CE3|AC5- |     |       |PE3|--|       |
       |    |AC6--0=====0--ENNI4|   |  +-------+
       +----+     |     |       +---+
                  +-----+

   Figure 5: Fast Convergence Upon ENNI Failure

				]]></artwork>
			<postamble></postamble>
                  +-----+]]></artwork>
</figure>

        <t>
   As discussed in <xref target="grouping"/>, it is highly desirable to have a
   mass withdraw
   mass-withdraw mechanism similar to the one in <xref target="RFC7432"/> . target="RFC7432"/>. Although
   such an optimization is desirable, it is OPTIONAL. <bcp14>OPTIONAL</bcp14>. If the optimization is
   implemented, the following describes the procedure:

            <list style="numbers"> procedures are used:

        </t>
        <ol spacing="normal" type="1">
	  <li>
            <t> When a vES is configured, the PE advertises the Ethernet Segment ES route
   for this vES with a color that corresponds to the associated physical port. </t>
          </li>
          <li>
            <t> All receiving PEs within the redundancy group record this color and
   compile a list of vESes associated with it. </t>
          </li>
          <li>
            <t> Additionally, the PE advertises a Grouping Ethernet A-D per ES route for EVPN,
   and a Grouping B-MAC route for PBB-EVPN, which corresponds to the color and vES grouping. </t>
          </li>
          <li>
            <t> In the event of a port failure, such as an ENNI failure, the PE withdraws
   the previously advertised Grouping Ethernet A-D per ES route or Grouping B-MAC route associated
   with the failed port. The PE should prioritize sending these Grouping route withdrawal
   messages over the withdrawal of individual vES routes affected by the failure. For
   instance, as depicted in Figure 5, <xref target="fig-4"/>, when the physical port associated with ENNI3 fails
   on PE2, it withdraws the previously advertised Grouping Ethernet A-D per ES route.
   Upon receiving this withdrawal message, other multi-homing multihoming PEs (such as PE1 and PE3)
   recognize that the vESes associated with CE1 and CE3 are impacted, based on the associated
   color, and thus initiate the DF election procedure for these vESes. Furthermore, remote
   PEs (such as PE4), upon receiving this withdrawal message, remote PEs (such as PE4) initiate the failover procedure
   for the vESes associated with CE1 and CE3, CE3 and switch to the other PE for each vES
   redundancy group. </t>
          </li>
          <li>
            <t> On reception of Grouping Ethernet A-D per ES route or Grouping B-MAC route withdrawal,
   other PEs in the redundancy group initiate DF election procedures
   across all their affected vESes. </t>
          </li>
          <li>
            <t> The PE with the physical port failure (ENNI failure), failure) sends a
   vES route withdrawal for every impacted vES. The other PEs upon Upon
   receiving these messages, the other PEs clear up their BGP tables. It should be
   noted that the vES route withdrawal messages are not used for executing DF
   election procedures by the receiving PEs when Grouping Ethernet A-D per ES
   route or Grouping B-MAC route withdrawal has been previously received. </t>
               </list>
           </t>
          </li>
        </ol>
      </section>
    </section>

    <section title="Acknowledgements">
    <t>
   The authors would like to thank Mei Zhang, Jose Liste, and Luc&nbsp;Andre&nbsp;Burdet for their
   reviews of this document and feedback.
    </t>
    </section>

    <section title="Security Considerations">

    <section>
      <name>Security Considerations</name>
      <t>
   All the security considerations in <xref target="RFC7432"/> and <xref target="RFC7623"/> apply
   directly to this document because this document leverages the control
   and data plane procedures described in those documents.
      </t>
      <t>
   This document does not introduce any new security considerations
   beyond that of <xref target="RFC7432"/> and <xref target="RFC7623"/> because advertisements and the
   processing of Ethernet Segment route ES routes for vES in this document follows follow
   that of physical ES in those RFCs.
      </t>
    </section>

    <section title="IANA Considerations">
    <section>
      <name>IANA Considerations</name>
      <t>
    This document requests has no actions from IANA. IANA actions.
      </t>
    </section>
  </middle>

  <!--  *****BACK MATTER ***** -->

<back>
  <references title="Normative References">
    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.2119.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.8174.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.7432.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.7623.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7623.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.8214.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8214.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.9135.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9135.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.8365.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8365.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.9541.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9541.xml"/>
      </references>
  <references title="Informative References">
      <references>
        <name>Informative References</name>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.7209.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7209.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.8584.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8584.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.7080.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7080.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.7023.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7023.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.4377.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4377.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.6310.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6310.xml"/>
        <xi:include href="https://www.rfc-editor.org/refs/bibxml/reference.RFC.9252.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9252.xml"/>

        <reference anchor="MEF63"> anchor="MEF63" target="https://www.mef.net/resources/mef-6-3-subscriber-ethernet-service-definitions">
          <front>
    <title>[MEF6.3]: Subscriber
            <title>Subscriber Ethernet Services Definitions</title>
    <author initials="MEF" surname="Metro
            <author>
              <organization>Metro Ethernet Forum">
      <organization></organization> Forum</organization>
            </author>
            <date month="11" year="2019"/>
          </front>
	  <seriesInfo name="MEF" value="6.3"/>
	  <refcontent>MEF Standard</refcontent>
        </reference>
      </references>
    </references>

 <section numbered="false">
      <name>Acknowledgements</name>
      <t>
   The authors would like to thank <contact fullname="Mei Zhang"/>, <contact
   fullname="Jose Liste"/>, and <contact fullname="Luc André Burdet"/> for
   their reviews of this document and their feedback.
      </t>
 </section>

<!-- [rfced] Terminology

c) May "multi-homed" and "multi-homing"
be changed to "multihomed" and "multihoming" per common
use in the RFC series (and in particular, in RFCs 7432,
7623, and 8584)? Your reply to this will be applied to the
other documents in this cluster (9785 and 9786).
-->

<!-- [rfced] Abbreviations

c) Please let us know how we may expand the following term:

  - SH (in the title of Figure 1)
-->

  </back>
</rfc>