Commit Graph

52 Commits

Author SHA1 Message Date
Jung-uk Kim
cc8440a569 Fix a typo introduced in r306024. 2017-01-23 21:09:27 +00:00
Alan Somers
4195c7de24 Always null-terminate ccb_pathinq.(sim_vid|hba_vid|dev_name)
The sim_vid, hba_vid, and dev_name fields of struct ccb_pathinq are
fixed-length strings. AFAICT the only place they're read is in
sbin/camcontrol/camcontrol.c, which assumes they'll be null-terminated.
However, the kernel doesn't null-terminate them. A bunch of copy-pasted code
uses strncpy to write them, and doesn't guarantee null-termination. For at
least 4 drivers (mpr, mps, ciss, and hyperv), the hba_vid field actually
overflows. You can see the result by doing "camcontrol negotiate da0 -v".

This change null-terminates those fields everywhere they're set in the
kernel. It also shortens a few strings to ensure they'll fit within the
16-character field.

PR:		215474
Reported by:	Coverity
CID:		1009997 1010000 1010001 1010002 1010003 1010004 1010005
CID:		1331519 1010006 1215097 1010007 1288967 1010008 1306000
CID:		1211924 1010009 1010010 1010011 1010012 1010013 1010014
CID:		1147190 1010017 1010016 1010018 1216435 1010020 1010021
CID:		1010022 1009666 1018185 1010023 1010025 1010026 1010027
CID:		1010028 1010029 1010030 1010031 1010033 1018186 1018187
CID:		1010035 1010036 1010042 1010041 1010040 1010039
Reviewed by:	imp, sephe, slm
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D9037
Differential Revision:	https://reviews.freebsd.org/D9038
2017-01-04 20:26:42 +00:00
Kashyap D Desai
605a51e323 This patch upgrades driver version to 06.712.04.00-fbsd
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 13:05:43 +00:00
Kashyap D Desai
2d53b48503 This patch will add code to refire IOCTL commands after OCR.
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 13:04:40 +00:00
Kashyap D Desai
d993dd831d This patch will unblock SYNCHRONIZE_CACHE command to firmware, i.e. don't block the SYNCHRONIZE_CACHE command at driver instead of
passing it to firmware for all Gen3 controllers.
For Thunderbolt controller, keep the legacy behavior i.e. return the SYNCHRONIZE_CACHE command with success status from driver itself.

There is Sysctl parameter 'block_sync_cache' is provided to enable customers either to block/unblock these commands to facilitate
legacy behavior if there is a compatibility issue. Default value for module parameter is to unblock this command.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 13:03:43 +00:00
Kashyap D Desai
85c0a961fa Wait for AEN task to be completed(if in queue) before resetting the controller
and return without processing event in AEN thread, if controller reset is in progress.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 13:02:48 +00:00
Kashyap D Desai
8bb601ac8a This patch will add task management support in driver. Below is high level description:
If a SCSI IO times out, then before initiating OCR, now the driver will try to send a
target reset to the particular target for which the IO is timed out. If that also fails,
then the driver will initiate OCR.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 13:01:31 +00:00
Kashyap D Desai
e2e8afb12c Process outstanding reply descriptors from all the reply descriptor post queues before initiating OCR.
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:59:38 +00:00
Kashyap D Desai
c2a20ff95a Clean up reference to AEN command if abort AEN is succesful as the command is aborted.
Did the same by setting sc->aen_cmd = NULL when aborting AEN is successful.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:58:28 +00:00
Kashyap D Desai
8bc320add0 Update controller properties(read OCR capability bit) when MR_EVT_CTRL_PROP_CHANGED recieved.
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:56:53 +00:00
Kashyap D Desai
808517a4f9 Add sanity check in IO and IOCTL path not to process command further if controller is in
HW_CRITICAL_ERROR.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:55:01 +00:00
Kashyap D Desai
f9c630813e Use a variable to indicate Gen3 controllers and remove all PCI ids based
checks used for gen3 controllers.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:53:05 +00:00
Kashyap D Desai
731b75614d Problem statement:
MFI linked list in megaraid_sas driver is used for mfi-mpt pass-through commands.
This list can be corrupted due to many possible race conditions in driver and
eventually we may see kernel panic.

One example -
MFI frame is freed from calling process as driver send command via polling method and interrupt
for that command comes after driver free mfi frame (actually even after some other context reuse
the mfi frame). When driver receive MPT frame in ISR, driver will be using the index of MFI and
access that MFI frame and finally in-used MFI frames list will be corrupted.

High level description of new solution -
Free MFI and MPT command from same context.
Free both the command either from process (from where mfi-mpt pass-through was called) or from
ISR context. Do not split freeing of MFI and MPT, because it creates the race condition which
will do MFI/MPT list.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   Broadcom Limited/AVAGO Technologies
2016-11-29 12:49:20 +00:00
Ed Maste
6f70040b38 mrsas: update for sys/capability.h rename in r263232 2016-09-20 12:59:30 +00:00
Kashyap D Desai
8cd174a463 Added support for Avago/Broadcom Cutlass(12 Gbps- 16 port count) controllers.
Submitted by:   Sumit Saxena <Sumit.Saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO/BROADCOM Limited
2016-06-02 12:26:55 +00:00
Kashyap D Desai
ac624ec089 Version update patch.
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:24:39 +00:00
Kashyap D Desai
4bb0a4f048 Following bugs fixed as part of this patch:
.Kernel panic while collecting kdump (reported by Doug A.)
.NULL pointer dereference at sertain places
.Removed dead codes

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:21:50 +00:00
Kashyap D Desai
c620f351de Added supprot for Avago Intruder controller.
Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:18:12 +00:00
Kashyap D Desai
8071588d7a Implemented interrupt Config Hook in mrsas(4) to defer some of the tasks, like:
riegistering AEN, creating cdev.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:15:20 +00:00
Kashyap D Desai
3a3fc6cba6 This patch implements driver support for 1MB IO size.
NOTE:
The FreeBSD system currently restricts the MAX IO size to MAXPHYS which
in turn is 128KB. We tested the 1MB IO by converting the MAXPHYS to 1MB.

Following is the mail reference:
http://lists.freebsd.org/pipermail/freebsd-scsi/2015-January/006568.html

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:12:09 +00:00
Kashyap D Desai
a688fcd018 Similar to RAID map for Logical Drives, now JBOD map has been introduced for
JBODs. Driver has to sync the JBOD map with firmware and use sequence number
as a reference for JBOD FastPath I/O's.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:05:02 +00:00
Kashyap D Desai
f0c7594bf3 There was no ERROR handling for firmware command TIMEOUT. This patch
takes care of any firmware command timeout scenarios by initiating OCR.

Submitted by:   Sumit Saxena <sumit.saxena@broadcom.com>
Reviewed by:    Kashyap Desai <Kashyap.Desai@broadcom.com>
MFC after:  3 days
Sponsored by:   AVAGO Technologies
2016-05-13 12:00:46 +00:00
Pedro F. Giffuni
453130d9bf sys/dev: minor spelling fixes.
Most affect comments, very few have user-visible effects.
2016-05-03 03:41:25 +00:00
Pedro F. Giffuni
87d8fcc80e Replace 0 with NULL for pointers in misc. device drivers.
Found with devel/coccinelle.
2016-04-12 17:23:03 +00:00
Steven Hartland
5bae00d6b4 Fix NULL pointer dereferences
Fix NULL pointer dereferences identified as V522 by PVS-Studio.

MFC after:	1 week
Sponsored by:	Multiplay
2016-02-25 02:46:47 +00:00
Justin Hibbits
43cd61606b Replace several bus_alloc_resource() calls using default arguments with bus_alloc_resource_any()
Since these calls only use default arguments, bus_alloc_resource_any() is the
right call.

Differential Revision: https://reviews.freebsd.org/D5306
2016-02-19 03:37:56 +00:00
Kashyap D Desai
2f2163ab40 Counter part of mfi driver commit in mrsas
Switch from make_dev_alias to make_dev_alias_p since make_dev_alias_p can
gracefully fail if the /dev/megaraid_sas_ioctl_node symlink already exists.
This can happen if mfi(4) and mrsas(4) are both attached to cards and
providing Linux emulation support.  Let the first one win.  An equivalent
change has been done in mfi(4).  Extra credit would be to pass the
Linux emulation call to the other driver when appropriate.  This will
probably be a rare case and the user can manually change where the symlink
points to.

MFC after:  3 days
2015-06-26 12:00:51 +00:00
Kashyap D Desai
5b2490f890 Kernel panic may be observed by user, if MR controller is under Chip reset (OCR)
and there are some pending IOs at the time of OCR. This is mainly because of
recursive mutext in OCR and IO completion function call. Generic IO completion (from ISR) needs
sim_lock to be held before it calls completion to CAM (xpt_done), but in case of OCR path mrsas_ocr thread
itself take sim_lock, so this condition is now handled in this patch.

MFC after:  3 days
2015-06-26 11:58:42 +00:00
Kashyap D Desai
dbcc81dfdc Corrected indentation on conflicted source files.
Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:46:28 +00:00
Kashyap D Desai
1bebd87179 Configured the mrsas(4) driver to support UNMAPPED I/O and updated driver version.
Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:45:56 +00:00
Kashyap D Desai
ecea5be464 1. All LSI namings are converted to AVAGO Tech.
2. Fix in AEN path(suggested by John Baldwin).
3. Fix IOCTL path w.r.t Sense key handling

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:45:13 +00:00
Kashyap D Desai
2f863eb8d8 Bug fixes found internally as detailed below:
1. While disabling interrupt the FW disables interrupts for only 16 vectors.
In case of Invader which supports 96 MSI-x vectors, some spurious interrupts
may come on other vectors even after interrupt disable. So, driver uses a flag
and ignores the spurious interrupts.
2. Reply queue depth is made double the number of commands supported by FW.
3. Misplaced interrupt enable code is now moved down in the OCR path.
4. Updated error handling code in OCR path.
5. Removed un-necessary print.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:44:08 +00:00
Kashyap D Desai
daeed97380 Driver calls mrsas_complete_cmd() to call mrsas_wakeup() for each MFI frame that was
issued through the ioctl() interface prior to the kill adapter. This ensures
userspace ioctl() system calls issued just before a kill adapter don't get stuck in
wait state and IOCTLs are returned to application.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:43:19 +00:00
Kashyap D Desai
af51c29f66 In OCR(Online Controller Reset) path, driver sets adapter state to
MEGASAS_HBA_OPERATIONAL before getting new RAID map.
There will be a small window where IO will come from OS with old RAID map.
This patch will update adapter state to MEGASAS_HBA_OPERATIONAL,
only after driver has new RAID map to avoid any IOs getting build using old RAID map.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:42:44 +00:00
Kashyap D Desai
16dc2814ee Current driver does fast path read load balancing between arm and mirror disk
for two Drive Raid-1 configuration only.

Now, Driver support fast path read load balancing for all (any number of disk) Raid-1 configuration.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:41:27 +00:00
Kashyap D Desai
77cf7df804 Now Driver expose Secure Jbod Support via driver_operations in MFI INIT Frame.
FW expose Secure Jbod support via Controller properity.

Firmware expect IOs to be received from different IO path than
conventional fast path queue, in case of SED drives.

To have Secure jbod support user need driver and firmware support.

Reviewed by:    ambrisko
MFC after:  2 weeks
Sponsored by:   AVAGO Technologies
2015-05-06 10:36:53 +00:00
Kashyap D Desai
5844115e3a This patch adds the feature to provide PCI information via IOCTL query.
Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2015-05-06 10:32:27 +00:00
John Baldwin
7fc5f32950 Use struct thread directly instead of d_thread_t. This driver is not
likely to be backported to 4.x.

Reviewed by:	kadesai
2015-01-06 15:41:23 +00:00
Steven Hartland
85c9dd9d89 Prevent overflow issues in timeout processing
Previously, any timeout value for which (timeout * hz) will overflow the
signed integer, will give weird results, since callout(9) routines will
convert negative values of ticks to '1'. For unsigned integer overflow we
will get sufficiently smaller timeout values than expected.

Switch from callout_reset, which requires conversion to int based ticks
to callout_reset_sbt to avoid this.

Also correct isci to correctly resolve ccb timeout.

This was based on the original work done by Eygene Ryabinkin
<rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid
the overlow.

Differential Revision:	https://reviews.freebsd.org/D1157
Reviewed by:	mav, davide
MFC after:	1 month
Sponsored by:	Multiplay
2014-11-21 21:01:24 +00:00
Hans Petter Selasky
f0188618f2 Fix multiple incorrect SYSCTL arguments in the kernel:
- Wrong integer type was specified.

- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.

- Logical OR where binary OR was expected.

- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.

- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.

- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.

- Updated "EXAMPLES" section in SYSCTL manual page.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2014-10-21 07:31:21 +00:00
Kashyap D Desai
f5fb223710 Fix for build issue on i386.xen and amd64.xen.
This is a duplicate variable reference in mrsas, so now this patch isolate atomic_ variable and relavent
function call using prefix mrsas_xx.

Issue was introduced in r272737.

MFC after:	2 weeks
Sponsored by:	Avago Technology
2014-10-13 15:33:27 +00:00
Kashyap D Desai
8e72737119 No logical code change in this pathc.
Only Style 9 changes for mrsas driver.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 10:14:37 +00:00
Kashyap D Desai
7efff25b10 Driver version upgrade.
Bring head mrsas same as internal Phase 6.5.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 09:39:18 +00:00
Kashyap D Desai
839ee02531 In the passthru IOCTL path, the mfi command pool was freely accessible N times
where as there are limited number(32) of mfi commands in the pool.
The mfi command pool is now restricted to 27 simultaneous accesses by using
a counting semaphore while calling the passthru function.

In the mrsas_cam.c source file there was a same function name mrsas_poll(),
which was same as the mrsas_poll() implemented in the mrsas.c file for the
polling interface.
To clearly distinguish the functionality by usage we have renamed the former
as mrsas_cam_poll().

In the passthru function let's say it has got an mfi command from the pool
but it has failed in one of the DMA function call which will lead to leak
an mfi command because in the ERROR case it directly returns and not freeing up
the occupied mfi command.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 09:37:47 +00:00
Kashyap D Desai
da01111344 d_poll() callback function is the entry point for poll system call for the application.
It is meant to notify the applications which will be waiting for some
controller events to be occured.

Reviewed by:	ambrisko
MFC after:	2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 09:35:52 +00:00
Kashyap D Desai
d18d1b472d Extended MSI-x vectors support for Invader and Fury(12Gb/s HBA).
This Driver will create multiple MSI-x vector depending upon what FW expose.
As of now 12 Gbp/s MR controller (Invader and Fury) expose 96 msix vector.
As of now 6 Gbp/s MR controller (Thunderbolt) expose 16 msix vector.

Reviewed by:	ambrisko
MFC after:		2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 09:34:25 +00:00
Kashyap D Desai
90012054d6 Fix the minor svn add issue. $FreeBSD$ expands at the time of
snv add, so I have added $FreeBSD$ as comment.

This commit is contininous of last mrsas commit, so that compilation
does not break.

Obtained from:	AVAGO Technologies
MFC after: 2 weeks
2014-10-08 09:30:35 +00:00
Kashyap D Desai
536094dc79 This is a feature provided to run 32-bit linux binaries on FreeBSD 64bit
machine, for which 32bit compatibilty code has been added.
As in linux there is only one device entry that is used to fire IOCTL commands,
a new device entry megaraid_sas_ioctl_node is added for solely this
purpose.

From one dev node i.e mrgaraid_sa_ioctl_node we have to find out the
controller instance in case of multicontroller, for which one management info
structure has been added.

Reviewed by:	ambrisko
MFC after:		2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 09:19:35 +00:00
Kashyap D Desai
4799d48568 Current MegaRAID firmware and hence the driver only supported 64VDs.
E.g: If the user wants to create more than 64VD on a controller,
    it is not possible on current firmware/driver.

New feature and requirement to support upto 256VD, firmware/driver/apps need changes.
In addition to that, there must be a backward compatibility of the new driver with the
older firmware and vice versa.

RAID map is the interface between Driver and FW to fetch all required
fields(attributes) for each Virtual Drives.
In the earlier design driver was using the FW copy of RAID map where as
in the new design the Driver will keep the RAID map copy of its own; on which
it will operate for any raid map access in fast path.

Local driver raid map copy will provide ease of access through out the code
and provide generic interface for future FW raid map changes.

For the backward compatibility driver will notify FW that it supports 256VD
to the FW in driver capability field.
Based on the controller properly returned by the FW, the Driver will know
whether it supports 256VD or not and will copy the RAID map accordingly.

At any given time, driver will always have old or new Raid map.

Reviewed by	:	ambrisko
MFC after	:	2 weeks
Sponsored by:	AVAGO Technologies
2014-10-08 08:48:18 +00:00
Kashyap D Desai
49328cc39c Fix for WITNESS warning while doing xpt_rescan.
This happen when converting any JBOD to RAID or creating
any new RAID from Unconfigured Drives.

Without this fix, user may see below call trace if  WITNESS is enabled.

witness_warn() at witness_warn+0x4b5/frame 0xfffffe011f929a00
uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe011f929a70
malloc() at malloc+0x192/frame 0xfffffe011f929ac0
mrsas_bus_scan_sim() at mrsas_bus_scan_sim+0x32/frame 0xfffffe011f929af0
mrsas_aen_handler() at mrsas_aen_handler+0x11c/frame 0xfffffe011f929b20
taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe011f929b80
taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe011f929bb0
fork_exit() at fork_exit+0x84/frame 0xfffffe011f929bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011f929bf0

Submitted by:   kadesai
Reviewed by:    ambrisko
MFC after:      3 days
2014-09-02 18:32:41 +00:00