Commit 9ef3e4a4 authored by James Smart's avatar James Smart Committed by James Bottomley

[SCSI] fc_transport: fix sysfs deadlock on vport delete

When the vport attribute "delete" is used to delete the vport, sysfs
deadlocks waiting for the write to complete, which is waiting for the
sysfs teardown to complete. Moved this effort to a work_q element.

Took the opportunity to make some other cosmetic changes:
 - removed tabs in Doc file - replaced with expanded spaces
 - minor copyright text and author text updates
 - removed a bunch of trailing whitespace
Signed-off-by: default avatarJames Smart <James.Smart@emulex.com>
Signed-off-by: default avatarJames Bottomley <James.Bottomley@SteelEye.com>
parent bee4fe8e
...@@ -119,67 +119,67 @@ Vport Attributes: ...@@ -119,67 +119,67 @@ Vport Attributes:
The new fc_vport class object has the following attributes The new fc_vport class object has the following attributes
node_name: Read_Only node_name: Read_Only
The WWNN of the vport The WWNN of the vport
port_name: Read_Only port_name: Read_Only
The WWPN of the vport The WWPN of the vport
roles: Read_Only roles: Read_Only
Indicates the FC4 roles enabled on the vport. Indicates the FC4 roles enabled on the vport.
symbolic_name: Read_Write symbolic_name: Read_Write
A string, appended to the driver's symbolic port name string, which A string, appended to the driver's symbolic port name string, which
is registered with the switch to identify the vport. For example, is registered with the switch to identify the vport. For example,
a hypervisor could set this string to "Xen Domain 2 VM 5 Vport 2", a hypervisor could set this string to "Xen Domain 2 VM 5 Vport 2",
and this set of identifiers can be seen on switch management screens and this set of identifiers can be seen on switch management screens
to identify the port. to identify the port.
vport_delete: Write_Only vport_delete: Write_Only
When written with a "1", will tear down the vport. When written with a "1", will tear down the vport.
vport_disable: Write_Only vport_disable: Write_Only
When written with a "1", will transition the vport to a disabled. When written with a "1", will transition the vport to a disabled.
state. The vport will still be instantiated with the Linux kernel, state. The vport will still be instantiated with the Linux kernel,
but it will not be active on the FC link. but it will not be active on the FC link.
When written with a "0", will enable the vport. When written with a "0", will enable the vport.
vport_last_state: Read_Only vport_last_state: Read_Only
Indicates the previous state of the vport. See the section below on Indicates the previous state of the vport. See the section below on
"Vport States". "Vport States".
vport_state: Read_Only vport_state: Read_Only
Indicates the state of the vport. See the section below on Indicates the state of the vport. See the section below on
"Vport States". "Vport States".
vport_type: Read_Only vport_type: Read_Only
Reflects the FC mechanism used to create the virtual port. Reflects the FC mechanism used to create the virtual port.
Only NPIV is supported currently. Only NPIV is supported currently.
For the fc_host class object, the following attributes are added for vports: For the fc_host class object, the following attributes are added for vports:
max_npiv_vports: Read_Only max_npiv_vports: Read_Only
Indicates the maximum number of NPIV-based vports that the Indicates the maximum number of NPIV-based vports that the
driver/adapter can support on the fc_host. driver/adapter can support on the fc_host.
npiv_vports_inuse: Read_Only npiv_vports_inuse: Read_Only
Indicates how many NPIV-based vports have been instantiated on the Indicates how many NPIV-based vports have been instantiated on the
fc_host. fc_host.
vport_create: Write_Only vport_create: Write_Only
A "simple" create interface to instantiate a vport on an fc_host. A "simple" create interface to instantiate a vport on an fc_host.
A "<WWPN>:<WWNN>" string is written to the attribute. The transport A "<WWPN>:<WWNN>" string is written to the attribute. The transport
then instantiates the vport object and calls the LLDD to create the then instantiates the vport object and calls the LLDD to create the
vport with the role of FCP_Initiator. Each WWN is specified as 16 vport with the role of FCP_Initiator. Each WWN is specified as 16
hex characters and may *not* contain any prefixes (e.g. 0x, x, etc). hex characters and may *not* contain any prefixes (e.g. 0x, x, etc).
vport_delete: Write_Only vport_delete: Write_Only
A "simple" delete interface to teardown a vport. A "<WWPN>:<WWNN>" A "simple" delete interface to teardown a vport. A "<WWPN>:<WWNN>"
string is written to the attribute. The transport will locate the string is written to the attribute. The transport will locate the
vport on the fc_host with the same WWNs and tear it down. Each WWN vport on the fc_host with the same WWNs and tear it down. Each WWN
is specified as 16 hex characters and may *not* contain any prefixes is specified as 16 hex characters and may *not* contain any prefixes
(e.g. 0x, x, etc). (e.g. 0x, x, etc).
Vport States: Vport States:
...@@ -198,23 +198,23 @@ Vport States: ...@@ -198,23 +198,23 @@ Vport States:
Once a vport has been instantiated with the kernel/LLDD, a vport state Once a vport has been instantiated with the kernel/LLDD, a vport state
can be reported via the sysfs attribute. The following states exist: can be reported via the sysfs attribute. The following states exist:
FC_VPORT_UNKNOWN - Unknown FC_VPORT_UNKNOWN - Unknown
An temporary state, typically set only while the vport is being An temporary state, typically set only while the vport is being
instantiated with the kernel and LLDD. instantiated with the kernel and LLDD.
FC_VPORT_ACTIVE - Active FC_VPORT_ACTIVE - Active
The vport has been successfully been created on the FC link. The vport has been successfully been created on the FC link.
It is fully functional. It is fully functional.
FC_VPORT_DISABLED - Disabled FC_VPORT_DISABLED - Disabled
The vport instantiated, but "disabled". The vport is not instantiated The vport instantiated, but "disabled". The vport is not instantiated
on the FC link. This is equivalent to a physical port with the on the FC link. This is equivalent to a physical port with the
link "down". link "down".
FC_VPORT_LINKDOWN - Linkdown FC_VPORT_LINKDOWN - Linkdown
The vport is not operational as the physical link is not operational. The vport is not operational as the physical link is not operational.
FC_VPORT_INITIALIZING - Initializing FC_VPORT_INITIALIZING - Initializing
The vport is in the process of instantiating on the FC link. The vport is in the process of instantiating on the FC link.
The LLDD will set this state just prior to starting the ELS traffic The LLDD will set this state just prior to starting the ELS traffic
to create the vport. This state will persist until the vport is to create the vport. This state will persist until the vport is
...@@ -222,65 +222,65 @@ Vport States: ...@@ -222,65 +222,65 @@ Vport States:
(state is one of the values below). As this state is transitory, (state is one of the values below). As this state is transitory,
it will not be preserved in the "vport_last_state". it will not be preserved in the "vport_last_state".
FC_VPORT_NO_FABRIC_SUPP - No Fabric Support FC_VPORT_NO_FABRIC_SUPP - No Fabric Support
The vport is not operational. One of the following conditions were The vport is not operational. One of the following conditions were
encountered: encountered:
- The FC topology is not Point-to-Point - The FC topology is not Point-to-Point
- The FC port is not connected to an F_Port - The FC port is not connected to an F_Port
- The F_Port has indicated that NPIV is not supported. - The F_Port has indicated that NPIV is not supported.
FC_VPORT_NO_FABRIC_RSCS - No Fabric Resources FC_VPORT_NO_FABRIC_RSCS - No Fabric Resources
The vport is not operational. The Fabric failed FDISC with a status The vport is not operational. The Fabric failed FDISC with a status
indicating that it does not have sufficient resources to complete indicating that it does not have sufficient resources to complete
the operation. the operation.
FC_VPORT_FABRIC_LOGOUT - Fabric Logout FC_VPORT_FABRIC_LOGOUT - Fabric Logout
The vport is not operational. The Fabric has LOGO'd the N_Port_ID The vport is not operational. The Fabric has LOGO'd the N_Port_ID
associated with the vport. associated with the vport.
FC_VPORT_FABRIC_REJ_WWN - Fabric Rejected WWN FC_VPORT_FABRIC_REJ_WWN - Fabric Rejected WWN
The vport is not operational. The Fabric failed FDISC with a status The vport is not operational. The Fabric failed FDISC with a status
indicating that the WWN's are not valid. indicating that the WWN's are not valid.
FC_VPORT_FAILED - VPort Failed FC_VPORT_FAILED - VPort Failed
The vport is not operational. This is a catchall for all other The vport is not operational. This is a catchall for all other
error conditions. error conditions.
The following state table indicates the different state transitions: The following state table indicates the different state transitions:
State Event New State State Event New State
-------------------------------------------------------------------- --------------------------------------------------------------------
n/a Initialization Unknown n/a Initialization Unknown
Unknown: Link Down Linkdown Unknown: Link Down Linkdown
Link Up & Loop No Fabric Support Link Up & Loop No Fabric Support
Link Up & no Fabric No Fabric Support Link Up & no Fabric No Fabric Support
Link Up & FLOGI response No Fabric Support Link Up & FLOGI response No Fabric Support
indicates no NPIV support indicates no NPIV support
Link Up & FDISC being sent Initializing Link Up & FDISC being sent Initializing
Disable request Disable Disable request Disable
Linkdown: Link Up Unknown Linkdown: Link Up Unknown
Initializing: FDISC ACC Active Initializing: FDISC ACC Active
FDISC LS_RJT w/ no resources No Fabric Resources FDISC LS_RJT w/ no resources No Fabric Resources
FDISC LS_RJT w/ invalid Fabric Rejected WWN FDISC LS_RJT w/ invalid Fabric Rejected WWN
pname or invalid nport_id pname or invalid nport_id
FDISC LS_RJT failed for Vport Failed FDISC LS_RJT failed for Vport Failed
other reasons other reasons
Link Down Linkdown Link Down Linkdown
Disable request Disable Disable request Disable
Disable: Enable request Unknown Disable: Enable request Unknown
Active: LOGO received from fabric Fabric Logout Active: LOGO received from fabric Fabric Logout
Link Down Linkdown Link Down Linkdown
Disable request Disable Disable request Disable
Fabric Logout: Link still up Unknown Fabric Logout: Link still up Unknown
The following 4 error states all have the same transitions: The following 4 error states all have the same transitions:
No Fabric Support: No Fabric Support:
No Fabric Resources: No Fabric Resources:
Fabric Rejected WWN: Fabric Rejected WWN:
Vport Failed: Vport Failed:
Disable request Disable Disable request Disable
Link goes down Linkdown Link goes down Linkdown
Transport <-> LLDD Interfaces : Transport <-> LLDD Interfaces :
...@@ -303,9 +303,9 @@ Vport Creation: ...@@ -303,9 +303,9 @@ Vport Creation:
int vport_create(struct fc_vport *vport, bool disable) int vport_create(struct fc_vport *vport, bool disable)
where: where:
vport: Is the newly allocated vport object vport: Is the newly allocated vport object
disable: If "true", the vport is to be created in a disabled stated. disable: If "true", the vport is to be created in a disabled stated.
If "false", the vport is to be enabled upon creation. If "false", the vport is to be enabled upon creation.
When a request is made to create a new vport (via sgio/netlink, or the When a request is made to create a new vport (via sgio/netlink, or the
vport_create fc_host attribute), the transport will validate that the LLDD vport_create fc_host attribute), the transport will validate that the LLDD
...@@ -342,7 +342,7 @@ Vport Creation: ...@@ -342,7 +342,7 @@ Vport Creation:
- Validate Infrastructure: - Validate Infrastructure:
- If the driver or adapter cannot support another vport, whether - If the driver or adapter cannot support another vport, whether
due to improper firmware, (a lie about) max_npiv, or a lack of due to improper firmware, (a lie about) max_npiv, or a lack of
some other resource - return VPCERR_UNSUPPORTED. some other resource - return VPCERR_UNSUPPORTED.
- If the driver validates the WWN's against those already active on - If the driver validates the WWN's against those already active on
the adapter and detects an overlap - return VPCERR_BAD_WWN. the adapter and detects an overlap - return VPCERR_BAD_WWN.
- If the driver detects the topology is loop, non-fabric, or the - If the driver detects the topology is loop, non-fabric, or the
...@@ -351,9 +351,9 @@ Vport Creation: ...@@ -351,9 +351,9 @@ Vport Creation:
of memory conditions, return the respective negative Exxx error code. of memory conditions, return the respective negative Exxx error code.
- If the role is FCP Initiator, the LLDD is to : - If the role is FCP Initiator, the LLDD is to :
- Call scsi_host_alloc() to allocate a scsi_host for the vport. - Call scsi_host_alloc() to allocate a scsi_host for the vport.
- Call scsi_add_host(new_shost, &vport->dev) to start the scsi_host - Call scsi_add_host(new_shost, &vport->dev) to start the scsi_host
and bind it as a child of the vport device. and bind it as a child of the vport device.
- Initializes the fc_host attribute values. - Initializes the fc_host attribute values.
- Kick of further vport state transitions based on the disable flag and - Kick of further vport state transitions based on the disable flag and
link state - and return success (zero). link state - and return success (zero).
...@@ -376,9 +376,9 @@ Vport Disable/Enable: ...@@ -376,9 +376,9 @@ Vport Disable/Enable:
int vport_disable(struct fc_vport *vport, bool disable) int vport_disable(struct fc_vport *vport, bool disable)
where: where:
vport: Is vport to to be enabled or disabled vport: Is vport to to be enabled or disabled
disable: If "true", the vport is to be disabled. disable: If "true", the vport is to be disabled.
If "false", the vport is to be enabled. If "false", the vport is to be enabled.
When a request is made to change the disabled state on a vport, the When a request is made to change the disabled state on a vport, the
transport will validate the request against the existing vport state. transport will validate the request against the existing vport state.
...@@ -404,7 +404,7 @@ Vport Deletion: ...@@ -404,7 +404,7 @@ Vport Deletion:
int vport_delete(struct fc_vport *vport) int vport_delete(struct fc_vport *vport)
where: where:
vport: Is vport to delete vport: Is vport to delete
When a request is made to delete a vport (via sgio/netlink, or via the When a request is made to delete a vport (via sgio/netlink, or via the
fc_host or fc_vport vport_delete attributes), the transport will call fc_host or fc_vport vport_delete attributes), the transport will call
......
/* /*
* FiberChannel transport specific attributes exported to sysfs. * FiberChannel transport specific attributes exported to sysfs.
* *
* Copyright (c) 2003 Silicon Graphics, Inc. All rights reserved. * Copyright (c) 2003 Silicon Graphics, Inc. All rights reserved.
...@@ -22,6 +22,7 @@ ...@@ -22,6 +22,7 @@
* Copyright (C) 2004-2007 James Smart, Emulex Corporation * Copyright (C) 2004-2007 James Smart, Emulex Corporation
* Rewrite for host, target, device, and remote port attributes, * Rewrite for host, target, device, and remote port attributes,
* statistics, and service functions... * statistics, and service functions...
* Add vports, etc
* *
*/ */
#include <linux/module.h> #include <linux/module.h>
...@@ -37,6 +38,7 @@ ...@@ -37,6 +38,7 @@
#include "scsi_priv.h" #include "scsi_priv.h"
static int fc_queue_work(struct Scsi_Host *, struct work_struct *); static int fc_queue_work(struct Scsi_Host *, struct work_struct *);
static void fc_vport_sched_delete(struct work_struct *work);
/* /*
* This is a temporary carrier for creating a vport. It will eventually * This is a temporary carrier for creating a vport. It will eventually
...@@ -377,7 +379,7 @@ static int fc_host_setup(struct transport_container *tc, struct device *dev, ...@@ -377,7 +379,7 @@ static int fc_host_setup(struct transport_container *tc, struct device *dev,
struct Scsi_Host *shost = dev_to_shost(dev); struct Scsi_Host *shost = dev_to_shost(dev);
struct fc_host_attrs *fc_host = shost_to_fc_host(shost); struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
/* /*
* Set default values easily detected by the midlayer as * Set default values easily detected by the midlayer as
* failure cases. The scsi lldd is responsible for initializing * failure cases. The scsi lldd is responsible for initializing
* all transport attributes to valid values per host. * all transport attributes to valid values per host.
...@@ -1198,12 +1200,9 @@ store_fc_vport_delete(struct class_device *cdev, const char *buf, ...@@ -1198,12 +1200,9 @@ store_fc_vport_delete(struct class_device *cdev, const char *buf,
size_t count) size_t count)
{ {
struct fc_vport *vport = transport_class_to_vport(cdev); struct fc_vport *vport = transport_class_to_vport(cdev);
int stat; struct Scsi_Host *shost = vport_to_shost(vport);
stat = fc_vport_terminate(vport);
if (stat)
return stat;
fc_queue_work(shost, &vport->vport_delete_work);
return count; return count;
} }
static FC_CLASS_DEVICE_ATTR(vport, vport_delete, S_IWUSR, static FC_CLASS_DEVICE_ATTR(vport, vport_delete, S_IWUSR,
...@@ -1996,7 +1995,7 @@ fc_attach_transport(struct fc_function_template *ft) ...@@ -1996,7 +1995,7 @@ fc_attach_transport(struct fc_function_template *ft)
i->t.eh_timed_out = fc_timed_out; i->t.eh_timed_out = fc_timed_out;
i->t.user_scan = fc_user_scan; i->t.user_scan = fc_user_scan;
/* /*
* Setup SCSI Target Attributes. * Setup SCSI Target Attributes.
*/ */
...@@ -2215,23 +2214,12 @@ fc_remove_host(struct Scsi_Host *shost) ...@@ -2215,23 +2214,12 @@ fc_remove_host(struct Scsi_Host *shost)
struct workqueue_struct *work_q; struct workqueue_struct *work_q;
struct fc_host_attrs *fc_host = shost_to_fc_host(shost); struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
unsigned long flags; unsigned long flags;
int stat;
spin_lock_irqsave(shost->host_lock, flags); spin_lock_irqsave(shost->host_lock, flags);
/* Remove any vports */ /* Remove any vports */
list_for_each_entry_safe(vport, next_vport, &fc_host->vports, peers) { list_for_each_entry_safe(vport, next_vport, &fc_host->vports, peers)
spin_unlock_irqrestore(shost->host_lock, flags); fc_queue_work(shost, &vport->vport_delete_work);
/* this must be called synchronously */
stat = fc_vport_terminate(vport);
spin_lock_irqsave(shost->host_lock, flags);
if (stat)
dev_printk(KERN_ERR, vport->dev.parent,
"%s: %s could not be deleted created via "
"shost%d channel %d\n", __FUNCTION__,
vport->dev.bus_id, vport->shost->host_no,
vport->channel);
}
/* Remove any remote ports */ /* Remove any remote ports */
list_for_each_entry_safe(rport, next_rport, list_for_each_entry_safe(rport, next_rport,
...@@ -2308,7 +2296,7 @@ fc_rport_final_delete(struct work_struct *work) ...@@ -2308,7 +2296,7 @@ fc_rport_final_delete(struct work_struct *work)
unsigned long flags; unsigned long flags;
/* /*
* if a scan is pending, flush the SCSI Host work_q so that * if a scan is pending, flush the SCSI Host work_q so that
* that we can reclaim the rport scan work element. * that we can reclaim the rport scan work element.
*/ */
if (rport->flags & FC_RPORT_SCAN_PENDING) if (rport->flags & FC_RPORT_SCAN_PENDING)
...@@ -2858,7 +2846,7 @@ EXPORT_SYMBOL(fc_remote_port_rolechg); ...@@ -2858,7 +2846,7 @@ EXPORT_SYMBOL(fc_remote_port_rolechg);
* fc_timeout_deleted_rport - Timeout handler for a deleted remote port, * fc_timeout_deleted_rport - Timeout handler for a deleted remote port,
* which we blocked, and has now failed to return * which we blocked, and has now failed to return
* in the allotted time. * in the allotted time.
* *
* @work: rport target that failed to reappear in the allotted time. * @work: rport target that failed to reappear in the allotted time.
**/ **/
static void static void
...@@ -3061,6 +3049,7 @@ fc_vport_create(struct Scsi_Host *shost, int channel, struct device *pdev, ...@@ -3061,6 +3049,7 @@ fc_vport_create(struct Scsi_Host *shost, int channel, struct device *pdev,
vport->shost = shost; vport->shost = shost;
vport->channel = channel; vport->channel = channel;
vport->flags = FC_VPORT_CREATING; vport->flags = FC_VPORT_CREATING;
INIT_WORK(&vport->vport_delete_work, fc_vport_sched_delete);
spin_lock_irqsave(shost->host_lock, flags); spin_lock_irqsave(shost->host_lock, flags);
...@@ -3207,8 +3196,30 @@ fc_vport_terminate(struct fc_vport *vport) ...@@ -3207,8 +3196,30 @@ fc_vport_terminate(struct fc_vport *vport)
} }
EXPORT_SYMBOL(fc_vport_terminate); EXPORT_SYMBOL(fc_vport_terminate);
/**
* fc_vport_sched_delete - workq-based delete request for a vport
*
* @work: vport to be deleted.
**/
static void
fc_vport_sched_delete(struct work_struct *work)
{
struct fc_vport *vport =
container_of(work, struct fc_vport, vport_delete_work);
int stat;
stat = fc_vport_terminate(vport);
if (stat)
dev_printk(KERN_ERR, vport->dev.parent,
"%s: %s could not be deleted created via "
"shost%d channel %d - error %d\n", __FUNCTION__,
vport->dev.bus_id, vport->shost->host_no,
vport->channel, stat);
}
MODULE_AUTHOR("Martin Hicks"); /* Original Author: Martin Hicks */
MODULE_AUTHOR("James Smart");
MODULE_DESCRIPTION("FC Transport Attributes"); MODULE_DESCRIPTION("FC Transport Attributes");
MODULE_LICENSE("GPL"); MODULE_LICENSE("GPL");
......
/* /*
* FiberChannel transport specific attributes exported to sysfs. * FiberChannel transport specific attributes exported to sysfs.
* *
* Copyright (c) 2003 Silicon Graphics, Inc. All rights reserved. * Copyright (c) 2003 Silicon Graphics, Inc. All rights reserved.
...@@ -104,7 +104,7 @@ enum fc_vport_state { ...@@ -104,7 +104,7 @@ enum fc_vport_state {
/* /*
* FC Classes of Service * FC Classes of Service
* Note: values are not enumerated, as they can be "or'd" together * Note: values are not enumerated, as they can be "or'd" together
* for reporting (e.g. report supported_classes). If you alter this list, * for reporting (e.g. report supported_classes). If you alter this list,
...@@ -117,7 +117,7 @@ enum fc_vport_state { ...@@ -117,7 +117,7 @@ enum fc_vport_state {
#define FC_COS_CLASS4 0x10 #define FC_COS_CLASS4 0x10
#define FC_COS_CLASS6 0x40 #define FC_COS_CLASS6 0x40
/* /*
* FC Port Speeds * FC Port Speeds
* Note: values are not enumerated, as they can be "or'd" together * Note: values are not enumerated, as they can be "or'd" together
* for reporting (e.g. report supported_speeds). If you alter this list, * for reporting (e.g. report supported_speeds). If you alter this list,
...@@ -223,6 +223,7 @@ struct fc_vport { ...@@ -223,6 +223,7 @@ struct fc_vport {
u8 flags; u8 flags;
struct list_head peers; struct list_head peers;
struct device dev; struct device dev;
struct work_struct vport_delete_work;
} __attribute__((aligned(sizeof(unsigned long)))); } __attribute__((aligned(sizeof(unsigned long))));
/* bit field values for struct fc_vport "flags" field: */ /* bit field values for struct fc_vport "flags" field: */
...@@ -397,7 +398,7 @@ struct fc_host_statistics { ...@@ -397,7 +398,7 @@ struct fc_host_statistics {
u64 prim_seq_protocol_err_count; u64 prim_seq_protocol_err_count;
u64 invalid_tx_word_count; u64 invalid_tx_word_count;
u64 invalid_crc_count; u64 invalid_crc_count;
/* fc4 statistics (only FCP supported currently) */ /* fc4 statistics (only FCP supported currently) */
u64 fcp_input_requests; u64 fcp_input_requests;
u64 fcp_output_requests; u64 fcp_output_requests;
...@@ -592,11 +593,11 @@ struct fc_function_template { ...@@ -592,11 +593,11 @@ struct fc_function_template {
u32 dd_fcrport_size; u32 dd_fcrport_size;
u32 dd_fcvport_size; u32 dd_fcvport_size;
/* /*
* The driver sets these to tell the transport class it * The driver sets these to tell the transport class it
* wants the attributes displayed in sysfs. If the show_ flag * wants the attributes displayed in sysfs. If the show_ flag
* is not set, the attribute will be private to the transport * is not set, the attribute will be private to the transport
* class * class
*/ */
/* remote port fixed attributes */ /* remote port fixed attributes */
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment